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INTRODUCTION 


The Large Area Crop Inventory Experiment (LACIE) is an interagency endeavor of- 
the National Aeronautics and Space Administration (NASA), the National Oceanic 
and Atmospheric Administration (NOAA), and the United States Department of 
Agriculture (USDA). Its purposes are (1). to demonstrate the economical bene- 
fit to be obtained by using remotely sensed data from the Land Satellite (Land- 
sat) for agricultural applications, (2) to test the capability of a system 
utilizing remote sensing in conjunction with climatological, meteorological, 
and conventional data to produce timely estimates of the production of a major 
world crop prior to harvest, and (3) to validate the technology and procedures 
for such a system. 

In accordance with the objectives of LACIE, the .Accuracy Assessment (AA) effort 
is designed to check the accuracy of the products from the experimental opera- 
tions throughout the growing season and thereby determine. if the procedures 
used are adequate to accomplish the above objectives. 

1.1 OBJECTIVES 

The objectives of AA ere as follows: 

a. To determine whether the accuracy goal of the LACIE estimate of wheat pro- 
duction for a region or country is being met. The LACIE accuracy goal is 
a 90/90 at-harvest criterion for wheat production. This specifies that 
the at-harvest wheat production estimate for the region dr country be 
within 10 percent of the true production with a probability of at least 0.9. 

b. To determine the accuracy and reliability of early season estimates and 
estimates made at regular intervals throughout a crop season prior to har- 
vest. This includes a determination of the degree to which the 90/90 cri- 
terion is supported at these intervals during the crop season. 

c. To investigate the various sources of error in’ the LACIE estimates of 
wheat production, area, and yield, to quantify and relate these error 
sources to causal elements in the LACIE estimation process, and to 
recommend procedures for reducing the error. 
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1.2 accuracy assessment activities 

In order to satisfy its objectives, AA carries out several types of evaluations 
and the results are presented in (1) monthly quick-look reports; (2) a number 
of interim reports leading up to a final report, and (3) certain special reports 
The following paragraphs contain the descriptions of the AA evaluations pre- 
sented in the three types of reports. 

1.2.1 ACTIVITIES REPORTED IN THE QUICK-LOOK REPORTS 

The quick-look reports contain an evaluation by AA of the LACIE estimates 
reported in the Crop Assessment Subsystem (CAS) monthly reports (CMR's) and 
the CAS unscheduled reports (CUR's). The quick-look reports are released one 
week following the release of a CMR or a CUR. The CMR's and CUR's contain 
the official LACIE estimates of wheat production, area, and yield, and the 
corresponding statistics. The true wheat production, area, and yield for the 
particular region or country are, of course, unknown. Therefore, to ascertain 
the accuracy of the -LACIE estimates, comparisons are made with a reference 
standard. In the United States, the reference standard consists of the most 
recent (at the time of the comparison) estimates released by the Statistical 
Reporting Service of the USDA (USDA/SRS). In'foreign countries, the reference 
consists of the most *"ecent estimates released by the Foreign Agricultural 
Service of the USDA (USDA/FAS). The AA quick-look reports contain a compari- 
son of the LACIE estimates of wheat production, area, and yield with the 
corresponding reference standard, as well as significance tests of no differ- 
ence at the region or country' level . The relative difference calculated at 
the zone level (state in the U.S.) is used to indicate problem areas; available 
blind site results are given and an intensive test site (ITS) example is 
presented . ' 

1.2.2 ACTIVITIES REPORTED IN THE INTERIM AND FINAL REPORTS 

The interim reports are released at regular intervals throughout the crop sea- 
son. They contain the results of the previous quick-look reports, a discus- 
sion of the 90/90 criterion as it applies to the region for which the LACIE 
estimates of wheat production are available, and the results of investigations 
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of error sources^ in the LACIE wheat production estimate including the blind 
site and ITS analyses. Also* any recommendations for improvement made by AA 
are documented in the interim and final reports. 

Each interim report is built up from the previous one by including data that 
became available during the interim period. Technical comments on each report 
are solicited from a variety of sources and are used to upgrade subsequent 
reports. Early- season and mid-season evaluations are made in the first and 
second interim reports; late-season and at-harvest evaluations are made in 
the third and fourth interim reports. 

The fourth interim report also serves as a draft for the final report, which 
contains material which is similar to the interim reports but covers the 
entire year. 

The above schedule. war. followed in Phase II. In Phase I there were no interim 
reports and the Phase I final report was incorporated into the Phase II final 
report. 


1.2.3 ACTIVITIES REPORTED IN AA UNSCHEDULED REPORTS 

From time to time, special investigations are carried out that are of interest’ 
to LACIE but which are not required on a regular basis such as those mentioned 
above. These investigations are reported in AA. unscheduled reports. 

1.3 PROCEDURES USED IN OBTAINING LACIE PHASE III ESTIMATES 

This report consists of evaluations of LACIE estimates of production, area, and 
yield for the U.S. Great Plains (US6P) region and for the U.S.S.R.; these esti- 
mates were released in the CAS reports for LACIE Phase III. During Phase III 
several changes were made in the aggregation procedures used by CAS. This 
section describes the procedures used in the various CAS reports. Some of the 
changes imposed by CAS during Phase III altered the Phase III monthly estimates. 


1 


A detailed description of the error sources in LACIE is given in appendix A. 
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The initial Phase III CAS report for the USGP was released February 8, 1977, 
prior to the availability of the Phase III allocation of sample segments. 

Thus, estimates published in the February CHR are the result of the aggrega- 
tion of segments from the LACIE Phase II allocation. 

On April 6, the second Phase III CAS report was released. The estimates in 
this report were based on the Phase III allocation, but only segments which 
were available as of the cutoff date of the February 8 CMR were used. Thus, 
the only difference between the results in the February 8 CMR and the April 6 
CUR is that in the fonner the Phase II allocation was used, whereas in the 
latter the Phase III allocation was used. The April 6 CUR was updated on 
April 22 with the release of a CUR which was based on all of the acquisitions 
from the Phase III allocation that were available at that date. 

In LACIE Phase III, C\S developed an objective thresholding procedure to 
eliminate acquisitions prior to emergence. This procedure was tested and 
was demonstrated to reduce the magnitude of the underestimate throughout the 
season. Thus, in addition to the regular estimates, CAS also generated the 
threshold' .estimates in the June and July CMR's. Further, the threshold esti- 
mate replaced the regular LACIE estimate in the August, September, and 
October CMR's. 

In September, CAS further modified the data with a procedure called screening, 
whereby segments were stratified according to historic county wheat propor- 
tions. In the screening procedure, CAMS proportion estimates which disagreed 
with their corresponding historic county proportions by a large margin 
(stipulated by the procecure) were excluded from the aggregation. 

As a result of the investigation of the overestimation problem in South Dakota, 
which was initiated immediately following the release of the July 11 CMR, a 
redesignation of sample segments into winter wheat, spring wheat, and mixed 
wheat segments was instituted in August' for the mixed wheat states of Montana 
and South Dakota. Previously, both winter and spring wheat estimates were 


1-4 



made for each segment In Montana and South Dakota* resulting in winter wheat 
estimates for segments containing little or no winter wheat and spring wheat 
estimates for segments containing little or no spring wheat. Under the redes- 
ignation, if a county containing allocated segments contributed 1 percent or 
more to the state winter wheat production, its segments were designated as 
winter wheat segments. The same rule applied for spring wheat. This divides 
the Counties into three groups:, pure winter, pure spring and mixed. Further, 
counties in the pure spring and pure winter wheat groups were subsequently ' 
designated mixed if the within-county contribution for either crop type to 
total wheat for the county was between 25 and 75 percent. This procedure was- 
also applied to' the oblasts in the mixed wheat region of the U.S.S.R. 

Table 1-1 is a summary of the procedures and allocations used in the various 
Phase III CHR's for the USGP. 

In the first U.S.S.R. CAS report, released on August 5, 1977, the estimates 
for production, area, and -yield'were obtained using the conventional aggrega- 
tion procedure. - 

In the second U.S.S.R. CMR, released on September 7, 1977, the official esti- 
mates were also obtained using the conventional procedure, but in addition 
some unofficial "modified" estimates were given.. The modified estimates were 
obtained using a procedure which was the same as the conventional procedure 
except that acquisitions obtained after Hay 1, 1977 were thresholded (i.e., 
not’ used) unless' prior acquisitions were also available, or unless usable 
acquisitions from biostage 6 or 7 were available. 

In the third U.S.S.R. CMR, released on October 5, 1977, the official estimates 
were obtained using the modified procedure. 
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TABLE 1-1.- CAS ALLOCATION AND PROCEDURE CHANGES FOR US6P DURING 

PHASE III OF LAC IE 


CAS Report Date 

A1 locati on 

Procedure 

February 8, 1977 

Phase II 

Conventional- 

April 6. 1977 

Phase III acquisitions 
available as of Feb- 
ruary 8 report 

Conventional 

April 22, 1977 

Phase III - all 
classifications avail- 
able to date 

Conventional 

June 7, 1977 

Phase III 

Conventional (official); 
Thresholding (unofficial) 

July 11, 1977 

Phase III 

Conventional (official); 
Thresholding (unofficial) 

August 10, 1977 

Phase III - Montana 
and South Dakota sites 
redesignated 

Thresholding (official) 

September 9, .1977 

Phase III - Montana 
and South Dakota sites 
redesignated 

Thresholding; screening 

October 11, 1977 

Phase III - USGP sites 
redesignated 

Thresholding; screening 
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1 . SUMMARY 


This report discusses the evaluations of the LACIE production, area, and 
yield estimates released in the February 7, April 6, April 22, May 9, June 7, 
duly n, August 10, September 9, and October 11 , 1977, CAS U.S. Great Plains 
reports. Also discussed are the estimates released in the CAS U.S.S.R. reports 
of August 5, September 7, and October 5, 1977. In the first three U.S. reports, 
the LAClE area estimates were compared with the USDA/SRS estimates of planted- 
area and the LACIE yield estimates were comparejd with a "derived'yield," 
obtained by dividing the USDA/SRS production estimate by the corresponding 
estimate of the planted area. The LACIE estimates released in. the May and 
later reports were compared with the corresponding USDA/SRS monthly estimates 
of harvested area, yield, and production. 

An accuracy of 90/85 was achieved with the October estimates which had a 
relative bias of -9.9 percent and a coefficient .of variation (CV) of 5.2 per- 
cent for the total wheat production in the USGP. That is, the probability is 
0,9 that the LACIE estimate was within ±15 percent of true wheat production 
for the USGP. 

The LACIE total wheat production estimates for the USGP region are available 
only in the August, September, and October CAS reports. In all three 
instances the LACIE estimate was significantly smaller than that of the 
USDA/SRS, primarily because the LACIE spring wheat production was under- 
estimated. All three USN6P spring wheat estimates were significantly smaller 
than their USDA/SRS counterparts, while there were no significant differences, 
between the LACIE and USDA/SRS USGP-7 winter wheat estimates from June (when 
LACIE statistics first became available) through October. 

The LACIE spring whear production underestimates in August, September, and 
October are the result of area underestimates for spring wheat in the USN6P 
region. The LACIE estimates were significantly smaller than the USDA/SRS 
estimates for all 3 months. 
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In the blind site investigation. It was found that for the USN6P spring wheat 
blind sites the average of the LACIE proportion estimates was significantly 
smaller at the 10-percent level than the average of the dot-count ground-truth 
wheat proportions. 

Winter wheat area estimates were generally in excess of their USDA/SRS 
counterparts. Small relative differences at the US6P-7 level in June and 
July resulted from overestimates in Colorado and -South Dakota, cancelling 
the underestimate in Oklahoma. The redesignation of segments eliminated the 
problem in South Dakota, and the thresholding procedure appears to have solved 
the underestimation problem in Oklahoma. A special investigation into the 
South Dakota winter wheat overestimate is contained in section 6.3 of this 
report. 

The winter wheat blind site study showed that the average proportion estimates 
are significantly different from the average dot-count ground- truth propor- 
tions at the USSGP ana USGP-7 levels. 

In Phase III, sampling appears to contribute slightly more to the variability 
of the area estimator than does classification; however, the CV for the 
total wheat area due to sampling in the USGP is only 1-9 percent, which is 
well within the sampling accuracy goal of 2.3 percent. Also, there is less 
•variability in the wi.iter wheat area estimates than in the spring wheat area 
estimates. 

The LACIE estimate of the total wheat yield for the USGP was consistently 
below that of the USDA/SRS with the relative difference varying between 
-9.9 and -11.3 percent during the crop year. This underestimate resulted 
from underestimates for both winter wheat (October relative difference = 

-9.0 percent) and spring wheat (October relative difference = -14.1 percent) 
for the USGP-7 and USWGP regions, respectively. The CV's for the LACIE total 
wheat (USGP) yield estimate were not available until October. A test of the 
difference between the October LACIE and USDA/SRS yield estimates for the 
USGP region showed that difference to be significant at the 10-percent level. 
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Jhe last section of this report consists of a comparison between the LACIE 
and the U.S.S.R. Task Force estimates of production, area, and yield for 
U.S.S.R. winter, spring, and total wheat. This analysis revealed steady 
improvement in the comparison of the production estimates during August, 
September, and Octobe". By October, there were no significant differences 
(at the 10-percent level) between the LACIE and the U.S.S.R. Task Force 
winter, spring, or total wheat production estimates. 
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3. ASSESSMENT OF PRODUCTION ESTIMATION 

This section contains an evaluation of LACIE performance relative to meeting 
the 90/90 criterion, it also includes a comparison of LACIE and USDA/SRS 
production estimates for winter wheat, spring wheat, and total, wheat. 

3 . 1 ' THE 90/90 CRITERION 

The LACIE accuracy goal for the USGP region is a 90/90 at-harvest criterion 
for wheat production. This specifies that the at-harvest wheat production 
estimate for the USGP region be within 10 percent of the true production with 
a probability of at least 0.90 for any given year. 


Let P be the LACIE at-harvest estimate of wheat production for the USGP and 
let P be the true wheat production for the USGP. The 90/90 criterion may be 
epcpressed by the following probability statement: 

,Pr[|P - Pj < O.IP] > 0.90 (3-1) 


It is reasonable to as.sume for large sample sizes that P is normally distnb- 

2 • . 

uted with mean P + B and variance op, where B is the bias of the estimator 
P. Under this assumption, it is shown in appendix A that equation (3-1) is 
equivalent to 


0.1 - 1.1 

CV(P) 


6 

P + B 


- $ 


-0.1 - 0.9 
CV(P) 


B 

P + B 


> 0.90 


(3-2) 


where $ represents the cumulative standard normal distribution and CV(P) is 
the CV of the estimator P defined by 


CV(P) = — ^ - — - 


E(P) 


P + B 


(3-3) 


The term 


B 

P + B 


is called the relative bias of P, 
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Inference as to whethe'^ the LACIE accuracy goal has been met is made by esti- 
Plating p ^ „ and CV(P) and then ascertaining whether equation (3-2) is satis- 

^ ° ^ Op 

fled. Now,’ CV(P) is estimated by where 5^ is an estimate of the standard 
deviation of P, and P is an unbiased .estimate of P + B. 

Assuming that the USDA/$RS wheat production estimate is the true wheat pro- 

R , P - P 

duction P, then p-jjrg- could be estimated simply by — 7 — 

With the October estimate of relative bias (-9.9 percent) and CV (5.2 percent), 
the 90/90 goal was not achieved'. However, an accuracy of 90/85 was achieved. 
That is, the probabil’ity that the LACIE estimate is within ±15 percent of the 
true wheat production for the USGP is 0.9. 

3.2 COMPARISON OF LACIE AND USDA/SRS PRODUCTION ESTIMATES 

Table 3-1 and figure 3-1 show how well LACIE performed relative to the USDA/ 

SRS estimates throughout the crop year. The nine dates for which data are 
provided correspond to the CAS reports of February 8 , April 6 , April 22, 

May 9, June 7, July 11, August 10, September 9, and October 11, 1977. Winter 
wheat estimates for the US6P-7 states (seven of the nine states of the USGP) 
are available for each of the above report dates, whereas spring wheat esti- 
mates for the four U.S, northern Great Plains (USNGP) states were generated 
only for the reports of August 10, September 9, and October 11, 1977. 

For each major region, a test was performed to determine if the LACIE esti- 
mate was significantly different from the corresponding USDA/SRS estimate. 

The test results are given in the last column of table 3-1. The testing pro- 
cedure used is described in appendix A. 

Because of software problems, statistics were not available for the LACIE pro- 
duction estimates until after the release of the May 9 CMR. Therefore, coef- 
ficients of variation (CV's) and tests of significance were available only for 

thnco octima+fic rplaa^pH af+Pr Mav Q. 1 Q77 
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TABLE 3-1 COMPARISON OF LACIE AND USOA/SRS PRODUCTION ESTIMATES 




Production 




Region 

n/M 

^USDA/SRS 

LACIE * 

Relative 

difference 

Value 

of 


Estimate 
(bu X 10^} 

cv 

{%) 

1976 CV 
(*) 

1977 

(*) 


RgyU 




February 8, 

1977 





WINTER WHEAT 

mm 








Colorado 


60280 

' 49772 

a 

33 

-21.1 



Kansas 


356400 

194220 

a 

17 

-83.5 



Nebraska 

31/35 

99000 

90058 

a 

23 

-9.9 



Okl ahoma 

27/40 

132600 

64391 

a 

29 

-105.9 

-41.1 


Texas 

34/49 

98400 

56762 

a 

28 

-73.5 

-26.9 



176/240 

746680 

455167 

a 

11 

-64.0 


a 

Montana 

25/60 

79300 

73799 

a 

a 

D 

a 


S. Dakota 

12/33 

13920 

28513 

a 

a 

BB 

a 


®HW states 



102312 

a 

a 


a 

a 

%GP-7 

215/333 

’839900 

557480 

a 

a 



a 

April 6, 1977 

WINTER WHEAT 









Colorado 

Z7/:Z 

60280 

48659 

a 

33 

-23.9 

37.0 


Kansas 

93/121 

356400 

187644 

a 

17 

-69.9 

-26.9 


Nebraska 

48/67 

99000 

88444 

a 

23 

-11.9 

39.2 


Okl ahoma 

40/46 

132600 

63918 

a 

29 

-107.5 

-41.1 


Texas 

27/38 

98400 

63305 

a 

28 

-55.4 

-26.9 


‘‘USSGP 

235/304 

746680 

451970 

a 

11 

-65.2 

-4.9 

a 

Montana 

40/80 

79300 

60723 

a 

a 

-30.6 

a 


S. Dakota 

22/56 

13920 

46978 

a 

a 

70.4 

a 


®HW states 

62/136 

93220 

107701 

a 

a 

13,4 

a 

a 

^USGP-7 

297/^40 

839900 

559672 

a 

a 

-50.1 

a 

a 


n » number of segments used. 

H 3 number of segments a1 located . 

®Data not available. 

^USDA/SRS prediction through April 22 
released on December 22, 1976. 

Relative difference - ^ 1 lQoj%. 

^U.S. southern Great Plains region. 

®The mixed wheat states, Montana 
^ and S. Dakota. 

^Seven-state winter whert region of 
U.S. Great Plains. 


*The LACIE estimate is signifi- 
cantly different from the USDA/SRS 
estimate at the 10- percent level. 

^The LACIE estimate Is not signifi- 
cantly different from the USOA/SRS 
estimate at the 10- percent level. 

^The pure spring wheat states, 
Minnesota and N. Dakota. 

1 . 

^U.S. northern Great Plains region. 

^U.5. Great Plains region. 


3-3 















































TABLE 3-1.- Continued 
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TABLE 3-U— Continued. 


Region 

n/M' 

V 

Proauction 

Cn T 

tive 

rence 

Value 

of 

‘^USDA/SRS 

LACIE 

Rela 

diffe 

Estimate 
{bu X 10^) 

Estimate 
(bu X 10^) 

CV 

{%) 

1976 CV 

i%) 

1977 

{%) 

1976 

{%) 

statistic 

June 7, 1977 

WINTER WHEAT 









Colorado 

28/32 

56640 

72456 

21.9 

28 

21.8 

31.7 


Kansas 

112/121 

396000 

308387 

11.5 

11 

-28.4 

14.4 


Nebraska 

50/67 

106750 

108793 

16.2 

17 

1.9 

24.4 


Oklahoma 

45/46 

169000 

96550 

14.0 

17 

-75.0 

-34.4 


Texas 

34/38 

110000 

91965 

14.2 

17 

-19.6 

16.5 


^USSGP 

269/304 

838390 

678151 

6.9 

7 

-23.6 

11.4 

-3.42* 

Montana 

41/80 

75600 

91417 

23.2 

192 

17.3 

-569.8 


S. Dakota 

28/56 

13600 

67685 

38.3 

46 

79,9 

34.1 


®MW states 

69/136 

89200 

159102 

21.1 

63 

43.9 

-147.1 

-2.08* 

%6P-7 

338/440 


837254 

7.0 

8 

-10.8 

1.7 

-1 .54*^ 

July 11, 1977 

WINTER WHEAT 




■ 


— 



Colorado 

30/32 

54280 

66516 


30 

18.4 

6.0 


Kansas 

111/121 

381300 

339348 

m 

11 

-12.4 

3.7 


Nebraska 

52/67 

106750 

111903 

15.7 

16 

4.6 

27.3 


Oklahoma 

42/46 

169000 

104907 

13.6 

'' 18 

-61.1 

-64.3 


Texas 

34/38 

115000 

91691 

13.9 

17 

-25.4 

-22.2 


'^USSGP 

269/304 

826330 

714365 

a 

7 

15.7 


a 

Montana 

58/80 

75600 

81983 

17.2 

53 

7.8 

-211.2 


S. Dakota 

39/56 

16320 

123196 

22.6 

27 

86.8 

63.1 


®MW states 

97/136 

91920 

205179 

a 

27 

55.2 

-46.7 

a 

%6P-7 

366/440 

91 8250 

919544 

6.4 

7 

0.1 ; 

-7.9 

0.02*^ 
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TABLE 3-1.— Continued 


Region 

n/M 

. Production 

^Relative 

difference 

Value 

of 

test 

statistic 

^USDA/SRS 

LACIE 

Estimate 
(bu X 10^) 

Estimate 
(bu X 10 ^) 

CV 

{%) 

1976 CV 
• (%) 

1977 

(%) 

1976 

W 




August 10, 

1977 





WINTER WHEAT 




■ 





Colorado 

31/32 

54280 

68682 

m 

29 

21.0 

3.2 


Kansa$ 

105/121 

350550 

357263 

HB 

10 

1.9 

3.1 


Nebraska 

40/67 

106750 

109960 

17.0 

16 

2.9 

26.5 


Okl ahoma 

44/46 

175500 

110463 

13.4 

18 

-58.9 

-54.0 


Texas 

33/38 

117500 

87579 

17.7 

18 

-34.2 

-28.2 


'*USSGP 

253/304 

804580 

733947 

6.8 

7 

-9.6 

my 

-1.41'^ 

Montana 

51/80 


72678 

|gS| 

36 

-4.0 

-73.2 


S. Dakota 

18/56 


36621 


26 

49.9 

56.2 


®MW states 

69/136 

93960 

109299 

17.6 

23 

14.0 

-15.4 

0.80^ 

■‘'uSGP-7 

322/440 

898540 

843247 

^9 

7 

-6.6 

-5.6 

-1.03^ 

SPRING WHEAT 

• 






■ 


Minnesota 

40/58 

130954 

71199 

18.1 

42 

-83.9 



N. Dakota 

63/103 

238250 

157751 

14.4 

17 

■ -51.0 

D 


®SW states 

103/161 

369204 

228950 

12.3 

16 

-61.3 

-40.4 

-4.98* 

Montana 

35/80 

50050 

24634 

22.8 

29 

: -103.2 

-116.2 


S. Dakota 

29/56 

58168 

45103 

18.3 

18 

-29.0 

44.6 


MW states 

65/136 

108218 

69737 

14.3 

17 

-55.2 

-26.6 

-3.86* 

•’USNGP 

167/297 

477422 

298686 

10.0 

13 

-59.8 

-37.8 

-5.98* 

TOTAL WHEAT 




■ 


■ 

■ 


Montana 

62/80 

125650 


m 

20 




S. Dakota 

38/56 

76528 


HI 

14 


B 

HIHI 

MW states 

100/136 

202178 


11.5 

12 

-12.9 

-19.8 

BBiB 


203/297 

571382 

407986 

9.0 

11 

-40.0 

-32.7 

-4.44* 

’USGP 

456/601 

1375962 

1141933 

5.4 

6 

-20.5 

-15.3 

-3.80* 











































































































TABLE 3-1 »— Continued 



■ WINTER WHEAT 

Colorado 

Kansas 

Nebraska 

Oklahoma 

Texas 


^USSGP 


Montana ■ 
S. Dakota 


®MW states 


%GP-7 


SPRING WHEAT 

Minnesota 
N. Dakota 


^SW states 


Montana 
S. Dakota 


MW states 


^USNGP 


TOTAL WHEAT 

Montana 
S. Dakota 


MW states 


USNGP 


60/136 


157/297 


53/80 

36/56 


89/136 


186/297 


430/601 


Production 

^USDA/SRS 

LACIE 

. Estimate 
(bu X iq3) 

Estimate 
(bu X iq3) 

CV 

{%) 

1976 CV 
{%) 



244/304 


39/80 

13/56 


52/136 


296/440 


37/58 
60/1 03 


54280 

350550 

106750 

175500 

117500 


804580 



96760 


901340 



97/161 359674 


48070 

55968 



104038 


463712 




26470 

74328 


•200798 


550472 


1365052 


September 9, 1977 


' 68675 
360616 
99264 
121671 
91594 


741820 


95206 

28130 


123336 


865156 


78744 

200529 


279273 


39357 

44969 


84326 


363599 


134563 

73098 


207661 


476935 


1228755 


''Rel ati ve 
difference 


1977 1976 

m {%) 



21.0 
2.8 
-7.5 13.5 

44.2 -56.7 

28.3 -27.2 


-8.5 -6.6 


21.5 -7.0 


.2 - 6.6 


66.3 -68.7 
14.1 -14.9 


28.8 -27.1 


22.1 -86.5 

24.5 32.3 


23.4 -26.4 


27.5 -27.0 


3.7 

15 

6. 

7.2 

13 

-1. 




3.3 -14.7 


-17.5 -22. 8‘ 


-11.1 -13.6 


Value 

of 

test 

statistic 






-2.48* 



- 1 . 86 * 


-2.93* 
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TABLE 3=-! Concluded 





'WINTER WHEAT 

Colorado 

Kansas 

Nebraska 

Oklahoma 

Texas 


Montana 
S. Dakota 


®MW states 


■'‘'uSGP-7 


SPRING WHEAT 

Mi nnesota 
N. Dakota 


^SW states 


Montana 
S. Dakota 


MW states 


TOTAL WHEAT 

Montana 
S. Dakota 


MW states 


l/SNGP 


23/31 

108/121 

40/56 

39/46 

28/35 


56/79 


294/368 


37/47 

70/103 


33/48 

32/37 


65/85 


172/235 



95/118 


202/274 


440/557 


Production 

^USDA/SRS 

LACIt 

Estimate 
(bu 10^) 

Estimate 
(bu X 10^) 

CV 

{%) 

1976 CV 
{%) 


Relative 

difference 




54280 

350550 

106750 

175500 

117500 


238/289 804580 



96760 


901340 


124714 

229985 


October 11, 1977 


77070 

365465- 

106120 

119208 

92885 


760748 


90411 

26072 


116483 


877231 




29.6 8.5 

4.1 3.7 

0.6 13.5 

-47-2 -56.7 
-26,5 -27.2 








107/150 354699 


50665 

55968 


106633 


461332 


129065 

74328 


203393 


558092 


1362672 


284460 


38683 

39748 


78431 


362890 


129094 

65820 


194914 


479373 


1240121 



-70.3 -89.7 
-8.9 -10.1 


-24.7 -26.2 


-65.7 

31.9 


-36.0 -19.8 


-27.1 -24.9 




-56.9 

46.0 


-4.4 rll.7 


■16.4 -20.9 


-9.9 -12.8 


Of 

test 

statistic 




-0.47" 




-3.03* 


-2.98* 



-0.42" 


- 1 . 86 * 


-1.90* 
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LEGEND 

LACIE 

••■•••USDA/SRS 
W= WINTER WHEAT 
S = SPRING WHEAT 
T - TOTAL WHEAT 


Figure 3-1 LACIE- and USDA/SRS production estimates (bushels x 10^). 
(USDA/SRS estimates through April 22 released on December 22, 1976.) 
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At the U.S. southern Great Plains (U5SGP) level, the LACIE and USDA/SRS winter 
wheat production esti'^ates differed by more than 300 million bushels in Feb- 
ruary but converged steadily after July and differed by less than 44 million 
bushels in October. This trend is most obvious in figure 3-1. It is worth 
noting that the LACIE estimate has experienced steady growth since the first 
aggregation of the Phase III allocation on April 6, whereas the USDA/SRS fig- 
ure increased through June but decreased in July and August. The relative 
difference between the LACIE and USDA/SRS production estimates at the USSGP 
level has decreased in magnitude with each aggregation since April 6, the 
first aggregation usiig the Phase III allocation. Statistics were not avail- 
able for LACIE USSGP production estimates through May or for the month of 
July. The difference between the LACIE and USDA/SRS estimates of winter wheat 
production for the USSGP region was significant {at the 10-percent level) in 
June but not in August, September, or October. 

The LACIE and USDA/SRS production estimates for the US6P-7 region followed a 
pattern very similar to that of the USSGP estimates except in the month of 
July, when the winter wheat area for South Dakota was grossly overestimated 
by LACIE, resulting in alarge production overestimate. The problem was cor- 
rected in August when the LACIE estimate dropped- to approximately the June 
level and resumed its approach toward the higher USDA/SRS estimate in Septem- 
ber and October. Since the June 7 CMR (the first month with statistics avail- 
able) there has been no significant difference (at the 10-percent level) 
between the LACIE and USDA/SRS winter wheat production estimates for the 
USGP-7 region. 

The first LACIE estimates of 1977 spring wheat production were made available 
in the August 10 CMR. The relative difference between the LACIE and USDA/SRS 
estimates for the four-state USNGP spring wheat producing region decreased in 
magnitude in each successive CMR because of increases in the LACIE estimate 
and decreases in the USDA/SRS estimate. The difference between the LACIE and 
USDA/SRS USNGP spring wheat production estimates was significant at the 
10-percent level for each of the three spring wheat aggregations as a result 
of a large LACIE undei estimate (as compared to the USDA/SRS estimate) for 
Minnesota and moderate underestimates for the other three states. 
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The LACIE USGP total wheat production estimates in August, September, and 
October were significantly smaller than the corresponding USDA/SRS estimates 
as a result of the underestimate in the USNGP spring wheat region. The mag- 
nitude of the relative difference between the two estimates decreased steadily 
during the three reporting months because of increases in the LACIE estimate 
and decreases in the dSDA/SRS estimate. 

The CV's at regional levels for both winter and spring wheat production 
estimates gradually decreased, indicating improvement in accuracy of the 
LACIE production estimates. 
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4. ASSESSMENT OF AREA ESTIMATION 


Three major subjects are discussed in this section: (1) a comparison of LACIE 

and USOA/SRS wheat area estimates (section 4.1); (2) a blind site investigation 
of proportion estimation error (section 4.2); and (3) a discussion of classifi- 
cation and sampling errors (section 4.3). 

4,1 COMPARISON OF LACIE AND USDA/SRS AREA ESTIMATES 

The LACIE and USDA/SRS area estimates are shown in figure 4-1 and table 4-1. 
Since the statistics published in the February, April, and May CAS reports were 
in error because of a software problem, statistical inferences are not given 
here for the data in these reports. 

The LACIE winter wheat area estimate at the five-state USSGP level increased 
steadily during the season after recording a small decrease in the April 6 
estimate (the first estimate using the Phase III allocation). ' Large negative 
relative differences recorded in February and April are due to the comparison 
of LACIE estimates of harvestable winter wheat area with USDA/SRS estimates of 
planted winter wheat area. Since May, however, the relative difference between 
the two estimates has ranged from -12.0 to +3.5 percent, improving steadily 
over the 5-month period except for October. 

Included in figure 4-1 is a plot of the Oklahoma winter wheat area estimate, 
which recovered from a -164,1 percent relative difference in February to 
-9.2 percent at the end of the season in October. Before May, the relative 
difference was large because of low LACIE estimates and a high USDA/SRS esti- 
mate. The USDA/SRS estimate is expected to be high at this time because it 
is for planted (rather than harvested) wheat. In May, the relative differ- 
ence improved (to -44.3 percent) due to an increase in the LACIE estimate and 
a decrease in the USDA/SRS estimate. The decreased USDA/SRS estimate is an 
estimate of harvested wheat. This estimate remained the same for the rest of 
the season. The LACIE estimate steadily approached this USDA/SRS estimate from 
May until September and then decreased slightly in October, There was no sig- 
nificant difference (at the 10-percent level) between LACIE and USDA/SRS USSGP 
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TABLE 4-1.- COMPARISON OF LACIE AND USDA/SRS AREA ESTIMATES 


Region 

n/M 

Area 


tive 

pence 

Value 

of 

test 

^USDA/SRS 

LACIE 

Re 1 c 
diffe 

Estimate 
(ac X 10^) 

Estimate 
(ac X 10 ^) 

cv 

(S) 

1976 CV 
(2) 

1977 

(2) 

1976 

(2) 

statistic 




February 8, 

1977 





WINTER WHEAT 









Colorado 

21/3*^ 

2740 

2183 

a 

26 


20.0 


Kansas 

65/84 

13200 

6719 

a 

12 


-63.5 


Nebraska 

31/35 

3300 

2977 

a 

18 

-10.8 

24.4 


Oklahoma 

27/40 

7800 

2953 

a 

24 

-164.1 

-90.0 


Texas 

34/4E 

61 50 

2954 

a 

25 

-108.2 

-98.7 


‘‘USSGP 

178/240 

33190 


a 

9 

-86.6 

m 

a 

Montana 

25/60 


Hi 

a 

a 

-10.4 

a 


S. Dakota 

12/33 



a 

a 

-n.i 

a 


states 

37/93 

4210 

3807 

a 

a 

-10.6 

a 

a 

%GP-7 

215/3.13 

37400 

21594 

•a 

a 

-73.2 

a 

a 




April 6> 

1977 





WINTER WHEAT 









Colorado 

27/32 

mBM 

2135 

a 

26 

-28.3 

20.0 


Kansas v 

93/121 


6491 

a 

12 

-103.4 

-63.5 


Nebraska 

48/67 

mSm 

2892 

a 

18 

-14.1 

24.4 


Oklahoma 

40/46 


2943 

a 

24 

-165.0 

-90.0 


Texas 

1 27/38 


3294 

a 

25 

-86.7 

-98.7 


‘‘USSGP 

235/304 

33190 

17755 

a 

• 9 

-86.9 

BQ 

a 

Montana 

40/80 

3050 


a 

a 

-34.1 

a 


S. Dakota 

22/56* 

1160 


a 

a 

32.6 

a 


®HW states 

62/136 

4210 

3995 

a 

a 


a 

a 

^USGP-7 

297/440 

37400 

21750 

a 

a 

-72.0 

a 

a 


n - number of segments used. 

M " number of segments aUocated. 

®Oata not available. 

^USDA/SRS prediction through April 22 
released on December 22, 1976. 

. .. . r-. /LACIE - USDA/SRS 

Relative difference = ^ LT^CH — 

^U.S, southern Great Plains region. 

®The mixed wheat states, Montana 
and S. Dakota. 


*The LACIE estimate is signifi- 
cantly different from the USDA/SRS 
estimate at the 10-percent level. 

^The LACIE estimate is not signifi- 
cantly different from the USDA/SRS 
estimate at the 10-percent level. 

loojx. ^The pure spring wheat states, 
Minnesota and N. Dakota. 

U.S. northern Great Plains region. 
^U.S. Great Plains region. 


"^Seven-state winter wheat region 
of U.S. Great Plains. 
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TABLE 4-1 Continued 




Area 



Value 

of 

test 

Region 

n/M 

^USDA/SRS 

LACIE 

Relative 

difference 



Estimate 
{ac X 10^) 

Estimate 
(ac X 10^) 

CV 

(2) 

1976 CV 
(2) 

1977 

(2) 

1976 

(2) 

statistic 




April 22, 

1977 





WINTER WHEAT 




■ 





Colorado 

27/32 

2740 

2189 


26 

-25.2 

20.0 


Kansas 

94/121 

13200 

6794 

H 

12 

-94.3 

-63.5 


Nebraska 

48/67 

3300 

3072 

II 

18 

-7.4 

24.4 


Oklahoma 

41/46 

7800 

3061 

a 

24 

-154.8 

-90.0 


Texas 

29/38 

6150 

3517 

H 

25 

-74.9 

-98.7 


‘^USSGP 

239/304 

33190 

18633 

a 

9 

-78,1 

-46.0 

a 

Montana 

40/80 



mm 

a 

-34.1 

a 


S. Dakota 

22/56 



B 

a 

32.6- 

a 


®MW states 

62/136 

4210 

|||||||||[|^^H 

a 

a 

Bl 

a 

a 

^USGP-7 


37400 

22627 

a 


-65.3 

a 

a 

Hay 9, 1977 

WINTER WHEAT 








— 

Colorado 

28/32 

2290 

3093 


24 

26.0 

32.3 


Kansas 

109/121 


10190 


6 

-17.8 

-15.0 


Nebraska 

48/67 


3169 


13 

3.8 

19.2 


Oklahoma 

45/46 


4506 


16 

-44.3 

-48,8 


Texas 

34/38 


4262 


14 

-3,2 

18.9 


‘^USSGP 


28240 

25220 

a 

6 

-12.0 

-3.2 

a 

Montana 


2800 

2973 

a 

a 

5.8 

a 


S. Dakota 

24/56 

750 

2261 

a 

a 

66,8 

a 


®MW states 

65/136 

3550 

5234 

a 

a 

32.2 

a 

a 

^USGP-7 

329/440 

31 790 

30453 

a ‘ 

a 

-4,4 

a 

a 


U3 









































































lAbLh 4-1 Lontinuea 



WINTER WHEAT 

Colorado 

Kansas 

Nebraska 

Oklahoma 

Texas 


Montana 
S. Dakota 


®HW states 



^USGP-7 


269/304 


58/80 

39/56 


97/136 


366/440 


1 Area 

*^USDA/SRS 

UCIE 

Estimate 
(ac X 10^) 

Estimate 
(ac X 10^) 

•cv 

i%) 

1976 CV 
{%) 


‘^Relative Value 

difference of 

test 

^ I statistic 

1977 : 1976 

{%) \ (X) 


WINTER WHEAT 


Colorado 

28/32 

Kansas 

112/121 

Nebraska 

50/67 

Oklahoma 

45/46 

Texas 

34/38 

'*USSGP 

269/304 

Montana 

41/80 

S. Dakota 

28/56 

®MW states 

69/136 

'•'uSGP-7 

338/440 


3480 


31790 


2360 

12300 

3050 

6500 

4600 


3480 


32290 


dune 7, 1977 



5854 


32848 


July 11, 1977 



2962 

13.2 

25 

20.3 

11764 

5.0 

6 

-4.6 

3475 

12.4 

11 

12,2 

5264 

8.5 

15 

-23.5 

4511 

11.6 

15 

-2.0 

27976 

3.9 

5 

-3.0 

3097 

12.3 

52 

9.6 

4629 

12.6 

23 

85.3 

7726 

9.0 

25 

55.0 

35701 

3.6 

5 

'9.6 


23.3 



6 . 11 * 


2.67* 

























































































TABLE 4-1 Continued. 



WINTER WHEAT 

Col orado 
Kansas 
Nebraska 
Okl ahoma 
Texas 


.Montana 
S. Dakota 


*HW states 


SPRING WHEAT 

Minnesota 
N. Dakota 


®SW states 


Montana 
S. Dakota 


MW states 


TOTAL WHEAT 

Montana 
S. Dakota 


KM states. 


USNGP 


’USGP . 



Area 

“^USDA/SRS 

LACIE 

Estimate 
(ac X 10^) 

Estimate 
(ac X 10^) 

CV 

{%) 

1976 CV 
{%) 



253/304 


51/80 

18/56 


69/136 


322/440 


40/58 

63/103 


103/161 


35/80 

29/56 


65/136 


62/80 

38/56 


100/136 


203/297 


456/601 


2360 

12300 

3050 

6500 

4700 


3480 


32390 


7997 


20729 


49639 


August 10, 1977 


3059 

12385 

3423 

5543 

4311 



9 


32819 



7635 


16634 


45355 











^Relati ve 
di fference 

Value 

of 

test 

1977 

{%) 

1976 
' (%) 

statistic 










-41.5 -55.2 


-105.4 
5', 5 


-27.7 -32.4 


-37.6 -49.5 



-26.0 


-43.4 


-18.7 




-19.7 

l.OS*^ 

-5.0 

o.ss'^ 



-2.47* 


-6.06* 



-1.88* 


-2.85* 































































































TABLE 4-1 Continued. 



Montana 
S. Dakota 


®HW states 


^USGP-7 


SPRING WHEAT 

Minnesota 
N. Dakota - 


Montana 
S. Dakota 


MW states 


TOTAL WHEAT 

Montana 
S. Dakota 


MW states 


USHGP 


Area 

*^USDA/SRS 

. LAC IE 

Estimate 
(ac X 10^) 

Estimate 
(ac X 10^) 

CV 

{%) 

1976 CV 
{%) 


^Relative Value 

di f ference of 

test 

statistic 

1977 1976 

( 2 ) (%) 


WINTER WHEAT 

Col orado 
Kansas 
Nebraska 
Okl ahoma 
Texas 


25/32 

07/121 

44/67 

38/46 

30/38 


244/304 


39/80 

13/56 


52/136 


296/440 


53/80 

36/56 


89/136 


186/297 


430/601 


3480 


32390 


7997 


20729 


49639 


September 9; 1977 



4636 


33888 


- 6.2 


-43.6 

28.4 


-14.2 


1 

5 

■B 

-7 

.2 

1 

.29 



20123 


49375 


-50.0 

-19.6 


-25.9 


-79.3 

2.1 


-28.9 


-26.6 


-57.2 

12.9 


-21 .4 


-24.3 


-13.9 























































































































TABLE 4-1.— Concluded. 



Area 

■^USDA/SRS 

LACIE 

Estimate 
(ac X 10^) 

Estimate 
(ac X 10^) 

CV 

{%) 

1976 CV 
(%) 


"'Relative Value 

di fference of 

- test 

statistic 

1977 1976 

{%) {%) 


October 11. 1977 


23/31 

108/121' 

40/56 

39/46 

28/35 


238/289 


2360 

12300 

3050 

6500 

4700 



WINTER WHEAT 

Colorado 

Kansas 

Nebraska 

Oklahoma 

Texas 


Montana 
S. Dakota 


®MW states 


•'uSGP-7 


SPRING WHEAT 


Montana 
S. Dakota 


MW states 

95/118 

7997 

8438 

12.0 

8 

5.2 

-17.8 

0.43*^ 

USNGP 

202/274 

20729 

19900 


5 

-4.2 

-24.7 

-0.55*^ 

^USGP 


49639 

49857 


4 

0.4 

-14.1 

0.17*^ 
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winter wheat area estimates in the months for which statistics were available 
(June through October) . 

At the USGP-7 level, the LACIE winter wheat area estimate increased steadily 
through July, exceeding the corresponding USDA/SRS estimate before decreasing 
in August because of a drop in the winter wheat area estimate for the mixed 
wheat states (Montana and South Dakota). The increase of the LACIE estimate 
through July was due primarily to overestimation in South Dakota (see fig- 
ure 4-1). The LACIE area estimate for that state grew to almost seven times 
the corresponding USDA/SRS estimate in July. An investigation of this large 
overestimation problem was conducted and is reported in section 6.3. As a 
result of this investigation, the sample segments were redesignated by crop 
type. The redesignation of sample segments in the August aggregation reduced 
the LACIE estimate 70 percent and took the relative difference in South Dakota 
from its July level of 85.3 percent to 49.3 percent. 

The differences between the LACIE and USDA/SRS winter wheat area estimates for 
the US6P-7 region were significant at the 10-percent level in July (because of 
the -South Dakota overestimate) and also in October. 

The LACIE USNSP spring wheat area estimate was significantly different from 
that of the USDA/SRS for each of the three reporting periods (August, Septem- 
ber, and October CMR'“). Underestimates (as compared to USDA/SRS estimates) 
were recorded for each of the four states in each of the three CMR's, although 
an improvement in the comparison was recorded in the September CMR, possibly 
due to the screening procedure investigated by CAS in that aggregation. 

The LACIE and USDA/SRS total wheat area estimates of September and October were 
not significantly different at the 10-percent level, although those of August 
were significantly different. The improvement in September can be attributed 
primarily to the improvement in the LACIE spring wheat area estimate discussed 
above. The relative difference for the USGP total wheat area estimate 
decreased in magnitude from -9.4 percent in August to +0.4 percent in October. 
The area CV's at regional levels decreased gradually. This indicates improve- 
ment in the accuracy of the LACIE area estimates. 
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4.2 BLIND SITE INVESTIGATION OF PROPORTION ESTIMATION ERROR 

This section contains a discussion of the wheat proportion estimation error 
using the blind site wheat estimates and the corresponding dot-count ground- 
truth proportion estimates for harvested wheat obtained by sampling the ground 
truth at 400 specified dots (or pixels). 

4.2.1 PROPORTION ESTIMATION ERROR 

Blind site results for winter wheat and spring wheat are shown in figure 4-2 
and tables 4-2 and 4-3. The CAMS proportions used are from the April 22, 

July 11, and October 11, 1977, CAS reports. The estimates in these reports 
were chosen because they were the latest proportion estimates from each interim 
reporting period. Figure 4-2 shows plots of the proportion estimation error 
X - X versus the dot-count ground-truth proportion X, where X is the ratioed- 
down wheat proportion estimate. Plots for the US6P-7 winter wheat producing 
region are included for the April 22, July 11, and October 11 CAS reports. 

A plot for the USGP spring wheat producing region is included for the 

A 

October 11 CAS report. Points lying above the horizontal line X - X = 0 
.correspond to overestimates of wheat proportions and points lying below the 
line correspond to underestimates.. 

The tendency for CAMS to underestimate by a greater margin for segments with 
larger proportions of wheat is exhibited by the plots for both winter and 
spring wheat. It is evident, though, from the three winter wheat plots, that 
this tendency became less pronounced as the season progressed. This gradual 
improvement is due primarily to the. maturation and eventual harvest of the 
wheat crop, although those allocation and aggregation modifications described 
in section 1 of this report also improved the accuracy of the estimates. 

Table 4-2 contains the results of the statistical analysis of the data for the 
April '22 CUR, the July 11 CMR, and the October 11 CMR. The following factors 
are listed: 

.A. 

© The averaged wheat proportion estimate X 

@ The averaged dot-count ground-truth wheat proportion estimate X 
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TABLE 4-2 - WINTER WHEAT BLIND SITE RESULTS 


Region 

n/H 

X 

X 

D 


90% confidence 
limits for Pp 

April 22, 1977 

Colorado 

n/32 

14.4 

22.2 

-7.8 

1,9 

(-n.2,-4,4)* 

Kansas 

19/121 

16.0 

27.6 

-11.6 

2.6 

(-16.1.-7.1)* 

Nebraska 

lUW 

19.7 

16.3 

3.4 

2.8 

H.5,8.3)N 

Oklahoma 


17.9 

36.7 

-18.8 

KSi 

(-25.3,-12.3)* 

Texas 

5/38 

20.1 

30.2 

-10.2 

4.8 

(-20.4,0. 0)N 

USSGP 

68/304 

17.4 

26,2 

-8.8 1 

1,7 

(-11.6,-6.0)* 

Montana 

9/80 

mm 

14.3 

-6.4 

1.9 

(-9.9, -2.9)* 

South Dakota 

9/56 

1.6 

2.0 

-0.4 

0,8 

C-1.9,1.1)N 

USGP-7 

86/440 

14.7 

22.3 

warn 

1.4 

(-10.0,-5.4)* 

July 11, 1977 

Colorado 

11/32 

16.0 

21.4 

-5.4 


(-9.4, -1.4)* 

Kansas 

22/121 

21.4 

27.1 

-5.7 


IRBirfffHfff 

Nebraska 

19/67 

14.1 

15.1 

-1.0 


(-3.6,1.6)N 

Oklahoma 

16/46 

27.8 

36.0 

-8-2 

Di 

(-13.3, -3.1)* 

Texas 

6/38 

20.3 

25.8 

-5-5 


(-11.5,0.5)N 

USSGP 


20,1 

25.0 

-5.0 

1.1 

(-6.8,-3.2)* 

Montana 

mmm 

ID 

WBBm 

-2.8 

1-1 

(-4.8,-0.8)* 

South Dakota 

5/56 



0.8 

1,2 

(-1.8,3.4)N 

USGP-7 

92/440 

17.9 

22.3 

mgm 

0.9 

(-5.9, -2.9)* 

October 11, 1977 

Colorado 

9/31 

18.6 

‘ 22,0 

-3.2 

2,0 

(-6.9,0.5)N 

Kansas 

21/121 

26.3 

29.1 

-2.8 

1.0 

; (-4.5,-I.D* 

Nebraska 

16/56 


17.5 


1.3 

mSBSBaSS^M 

Oklahoma 

14/46 

34.8 

38.2 

-3.4 

2.8' 

\ (-8.4,1 .6)N 

Texas 

6/35 

21 .6 

25.8 

-4.2 

2.3 

; (-8.8,0.4)N 

USSGP 


24,0 

fiiTT” 

-3.0 

0.8 

(-4.3,-1,7)* 

KSlSSSiliHI 


13.5 

J3.8 

-0.3 



South Dakota 

3/21 

3.0 


IIQQII 



USGP-7 

83/368 

21.5 

23.8 

■WMI 

0.7 

[ (-3.6, -1.2)* 


LEGEND: 


n - number of blind sites available 
H = number of sample segments allocated 

X = average of wheat proportion estimates 

X = average of dot-count ground-truth wheat proportion estimates for 
harvested wheat 

D = I- X 

Sg- = standard error of 0 
iijj = population d 

N = Up not significantly different from zero at the 10-percent level 
* = Up significantly different from zero at the 10-percent level 
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TABLE 4-3.- SPRING WHEAT BLIND SITE RESULTS 
[October 11, 1977. CMR] 


Region 

n/M 

X 

X 

D 

D 

90% confidence 
limits for Up 

Minnesota' 

13/47 

18.2 

21.7 

-3.5 

2.3 

(-7, 6,0.6)^ 

North Dakota 

20/1 03 

21.0 

25.1 

-4.1 

1.5 

(-6;7,-1.5)* 

Montana 


11.2 

14.6 

-3.4 

2.1 

(-7. 2, 0.4)^ 

South Dakota 

10/37 

8.4 

11.0 

-2.6 

2.2 

(-6. 6,1. 4)*^ 

USNGP(S) 

53/235 

16.1 

19.6 

-3.5 

1.0 

(-5. 2, -1.8)* 


LEGEND: 

* = Uq significantly different from zero at the 10-percent level 

N = yjj not significantly different from zero at the 10-percent level 

n = number of blind sites available 
M = number of sample segments allocated 

average of wheat proportion estimates 
X = average of dot-count ground-truth wheat proportion estimates for 
h_arvested wheat 

D = X - X 

Sg- = standard error of D 
Ug = population "D 
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e The averaged difference D = X - X 
9 The standard error of the averaged difference 

9 The 90-percent confidence limits for the population averaged difference 
The formulas for calculating these factors are given in appendix A. 

To determine whether the average' difference for a particular state or region 
is significantly different from zero, one may simply check whether the corre- 
sponding confidence interval contains zero. If it does, the averaged difference 
is not significantly different from zero; i.e., there is insufficient evidence 
to conclude that a bias exists due to proportion estimation error. If the 
confidence interval does not contain zero, the hypothesis for no bias is re- 
jected. The test is. performed at the 10-percent level of significance.. 

In the April 22 blind site results (table 4-2), the proportion estimation error 
was significantly different from zero in both the USSGP and USGP-7 regions and 
for all states of the US6P-7 region except Nebraska, Texas, and South Dakota. 

The average difference in Oklahoma was a particularly large negative value. 
ATlt-'averaged differences in the April 22 investigation were negative except 
for Nebraska. A negative average difference indicates that the average LACIE 
proportion estimate is less than the average dot-count ground- truth proportion 
estimate. 

Blind site results ui ttic uui^ ii mvcaLiyai.ion revealed average differences 
smaller in magnitude than those for April 22 in both the USSGP and USGP-7 
regions and ip every state except South Dakota. Earlier in this section, it . 
was noted that as the season progressed the tendency for LACIE to underestimate 
for segments with larger proportions of wheat was less pronounced. All average 
differences, except for South Dakota, were negative in the July 11 investiga- 
tion. The average differences were significantly different from zero in both 
regions and in all states except Nebraska, Texas, and South Dakota. 
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Results of the October winter wheat blind site investigation show that the 

/\ 

differences between X and X were again smaller in magnitude that those re- 
corded in July in both winter wheat regions and in every winter wheat state 
except Nebraska. The differences were significant only in the USSGP and 
US6P-7 regions and in the state of Kansas. All average differences were 
negative, indicating underestimation of wheat proportions. 

The large underestimation problem in Oklahoma, as shown in the blind site 
results for April 22 and July 11 data, .was remedied in the October 11 blind 
site results and is no longer considered significant. 

.Figure 4-2 contains plots for USGP spring wheat blind sites for the October 
CMR. The USNGP(S) plot shows the tendency for LACIE to underestimate by a 
greater margin for larger ground-truth proportions, similar to the winter 
wheat plots. 

Table 4-3 is a summary of the spring wheat blind site study. corresponding to 
the USNGP(S) plot. The average classification errors were negative for all 
four spring wheat states and were significantly different from zero in 
North Dakota and for the USNGP region. 
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4.3 SAMPLING AND CLASSIFICATION ERRORS 

This study was performed for the purpose of (1) measuring the contributions 
of classification and sampling errors to the within-stratum area variance and 
(2) estimating the CV's of the area estimates due to classification and sam- 
pling errors. 

To estimate the within-stratum area variances due to classification and sam- 
pling errors, one first constructs the following three basic regression 
models: 

e True segment proportion versus historical stratum proportion 

0 LACIE. segment proportion versus ground-truth segment proportion 

© LACIE segment proportion versus historical stratum proportion 

These regression models are used to obtain, respectively, the estimates for 
(!) the variance contribution due to sampling (often called sampling variance) 
and the variance of the residuals resulting from regressing the current true 
stratum proportion onto the historical stratum proportion, (2) the variance 
contribution due to classification (often called classification- variance) , 
and (3) the classification and sampling variances. The maximum likelihood 
estimation technique, assuming normality, is then used to obtain the optimal 
estimates for sampling and classification variances. A detailed description 
of this method is presented in appendix A (section A. 3. 1.5.1). 

When the above-mentioned variance estimates are obtained, the ratio p of the 
within-stratum sampling- variance estimate to the total within-stratum area 
variance estimate can be calculated easily. Assuming that this ratio applies 
to each 2 one and each higher region, the variances of the large area estimate 
due to classification and sampling are given by 

= (1 - p)v^ (4-1) 

and 

0^ = pV^ (4-2) 
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‘2 2 ^2 

where n , v , and V denote the classification variance, the sampling vari- 
ance, and -the area variance, respectively, for the large area estimate. Con- 

A 

sequently, the estimated CV of a‘ large area estimate A due to classification 
is given by 

CV(A/C) * ^ (4-3) 

A 

tuiu Lfie eitimaucu CV of a large area estimate due to sampling is given by 

CV{A/S) = i (4-4) 

A 

where CV(A/C) and CV(A/S) are often casually referred to as the classification 
CV and sampling CV, respectively. 


Estimates of these variances and CV*s for the October LACIE estimates are 
tabulated below. 



Within- 

Variance component 

Percentage error 

Classi- 
fication 
CV (%) 


Crop 

Stratum 

area 

variance 

Due to 
classi- 
fiv.ation 

Due to 
sampling 

B 

Due to 
sampling 

Sampl i ng 
CV ■{%} 

Winter 

wheat 

USGP-7 

104.1 

41.6 

62.5 

40 

60 

2.0 

2.5 

Spring 

wheat 

USNQP 

1 

! 

65.6 

26.2 

39.4 

40 

60 

2.3 

2.8 

Total 

wheat 

USGP 

100.4 

39.6 

60.8 

40 

60 

1.5 

1.9 


These results show that the. sampling CV is larger than the classification CV 
for winter, spring, and total wheat estimates. The indication is that sampling 
contributes slightly more error to the area variance than does classification. 
Moreover, winter wheat has smaller CV's for both classification and sampling 
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than does spring wheat; i.e., there is less variability in the winter wheat 
area estimates than in the spring wheat area estimates. For the USGP region, 
the sampling CV for the total wheat area estimate is 1.9 percent, which is 
well within the sampling accuracy goal of 2.3 percent. 
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5. ASSESSMENT OF YIELD ESTIMATION 


This section consist's of a comparison of the LACIE and USDA/SRS yield estimates 
and an assessment of the crop calendar model accuracy. 

5.1 COMPARISON OF LACIE AND USDA/SRS YIELD ESTIMATES 

Figure 5-1 and table 5-1 present the LACIE and USDA/SRS yield estimates. For 
the first three CAS reports of Phase III (February 8, April 6, and April 22, 
1977), no yield estimates were available from USDA/SRS. AA, for purposes of 
comparison, "derived" USDA/SRS yield estimates by dividing the USDA/SRS pro- 
duction estimates by the corresponding estimates of planted area. These yield 
estimates remained unchanged through April at both regional levels (USSGP and 
US6P-7) since revised estimates were not released by the USDA/SRS until May. 

The LACIE estimates of winter wheat yield for the USSGP and USGP-7 regions 
remained relatively constant throughout Phase III. The LACIE USSGP estimates 
ranged from 24.9 to 25.6 bushels per acre while the USGP-7 estimates ranged 
from 25.5 to 25.8 bushels per acre. LACIE estimates were consistently below 
the corresponding USDA/SRS estimates from May through October in both regions. 
The differences between the LACIE and USDA/SRS estimates of yield for the 
USSGP and USGP-7 regions were significant at the 10-percent level in every 
month for which statistics were available (June through October) except August. 

For the winter wheat states, relative differences for Oklahoma and Texas 
through Phase III indicated a large underestimate when compared to the USOA/ 

SRS estimates. The trend term problem and the questionable precipitation 
variable in the Center for Climatological and Environmental Assessment (CCEA) 
yield models, either together or individually, may have contributed to the 
large underestimation of yield. Specifically, the trend term, which depends 
on a multitude of factors including irrigation, has been assumed to be zero 
since 1960 (when the trend curve leveled off) for the Oklahoma and Texas/ 
Oklahoma panhandle models. However, irrigation practices (largely concen- 
trated in the panhandle) began in Texas after 1960. At any given time the 


5-1 



TABLE 5-1 •- COMPARISON OF LACIE AND USDA/SRS YIELD ESTIMATES 


Region 

Yield 

Cn 1 

tive 

rence 

Value 

of 

test 

'^USDA/SRS 

UCIE 

Rela 

diffe 

Estimate 

(bu/ac) 

Estimate 

(bu/ac) 

CV 

(*) 

1976 CV 
(2) 

1977 

(2) 

1976 

(«} 

statistic 



February 8, 

1977 




WINTER WHEAT 

■■ 







Colorado 

mM 

22.8 

a 

21 

3.5 

21.3 

• 

Kansas 

mm 

28.9 

a 

12 

6.6 

22.4 


Nebraska 

30,0 

30.2 

a 

14 

0.7 

19.6 


Oklahoma 

17.0 

21.8 

a 

■■ 

22.0 

3*4.5 


Texas 

16.0 

19.2 

a 

B9I 

16.7 

36.2 


‘*USSGP . 

22.5 

25.6 

a 

7 

12.1 

28.3 ' 

a ' 

Montana 

26.0 

26.7 

a 

a 

2.6 

a 


S. Dakota 

12.0 

27.3 

a 

a 

56.0 

a 


®HW states i 

22.1 

26.9 

a 

a 

17,8 

a 

a 

^USGP-7 


25.8 

a 

a 

12.8 

a 

a 



Apri 1 

6, 1977 



■ 

WINTER WHEAT 








Colorado 

22.0 

22.8 

a 

21 

3.5 

21.3 


Kansas 

27.0 

28.9 

a 

12 

6.6 

22.4 


Nebraska 

30.0 

30.6 

a 

14 

2.0 

19.6 


Oklahoma 

17.0 

21.7 

a 

mm 

21.7 

34.5 


Texas 

16.0 

19.2 

a 

■■ 

16.7 

36.2 


‘^USSGP 

22.5 

25.5 

a 

7 

; 11.8 

28.3 

a 

Montana 

MSM 

26,7 

a 

a 

2.6 

a 


S. Dakota 

m 

* 27.3 

a 

a 

56.0 

a 


®HW states 

22.1 

27.0 

a 

a 

18.1 

a 

a 

^USGP-7 

22.5 

25. 7 

a 

a 

12,5 

a 

a 


®Data not available. 

^USDA/SRS estimates through April 22 
derived from estimates of seeded 
acres and production released on 
December 22 , 1976. 

^Relative difference 

^U.S. southern Great Plains region. 

®The mixed wheat states, Montana 
and S. Dakota. 

^Seven-state winter wheat region 
of U.S. Great Plains. 


*The LACIE estimate is signifi- 
cantly different from the USDA/SRS 
estimate at the 10-percent level. - 

^he LACIE estimate Is not signifi- 
cantly different from the USDA/SRS 
estimate at the 10-percent level. 

^The pure spring wheat states, 
Minnesota and N. Dakota. 

^U.S. northern Great Plains region. 

^U.S. Great Plains region. 
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TABLE 5-1 Continued. 


Region 

Yield 

Cn 1 

itive 

irence 

Value 

of 

test 

^USDA/SRS 

LACIE 

Re la 
di f f € 

Estimate 

(bu/ac) 

Estimate 

(bu/ac) 

CV 

1976 CV 
{%) 

1977 

{%) 

B 

HH 

statistic 



April 22, 

1977 




WINTER WHEAT 








Colorado 

22.0 

22.4 

a 

21 

1,8 

21.3 


Kansas 

27.0 

28.1 

a 

12 

. 3.9 

22.4 


Nebraska 

30.0 > 

31.4 

a 

14 

4.5 

19.6 


Oklahoma 

17.0 

21.0 

a 

17 

19.0 

34.5 


Texas 

16.0 

18,1 

a 

19 

11.6 

36.2 


‘^USSGP 

22.5 

24.9 

a 

7 

9.6 

28.3 

a 

Montana 

MM 

28,9 

a 

a 

10.0 

a 


S. Dakota 

ml 

26.8 

a 

a 

55.2 

a 


®MW states 

22.1 

28.0 

a 

a 

21.1 

a 

a 

^US6P-7 

22.5 

25.5 

a 

a 

11.8 

a 

a 

May 9, 1977 

WINTER WHEAT 



mm 





Colorado 

24.0 

22.8 

B 

20 

-5.3 

-11.7 


Kansas 

32.0 

28.1 

B 

10 

-13.9 

7.'0 


Nebraska 

34.0 

31 .3 


14 

-8.6 

-6.0 


Oklahoma 

25.0 

21.2 


14 

-17.9 

3.2 


Texas 

23.0 

19.5 

B; 

13 

-17.9 

0,6 


‘^USSGP 

28.6 

25.2 

a 

6 

-13.5 

1.6 

a 

Montana 

27,0 

28,8 

a 

a 

6.3 

a 


S. Dakota 

20.0 

26.0 

a 

a 

23.1 

a 


®MW states 

25, f 

27,6 

a 

a 

7.6 

a 

a 

^USGP-7 

28.2 

25.6 

a 

a 

-10.2 

a 

a 
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TABLE 5-1 Continued 




Yield 


“USDA/SRS 


Estimate 

(bu/ac) 


LACIE 


Relative 

difference 


WINTER WHEAT 

Colorado 

Kansas 

Nebraska 

Oklahoma 

Texas 


Montana 
S. Dakota 


"MW states 


'USGP-7 


WINTER WHEAT 

Colorado 

Kansas 

•Nebraska 

Oklahoma 

Texas 



Estimate 

CV 

(bu/ac) 

{%) 

Oune 

7, 1 

23.6 

16.9 

• 28.3 

10.6 

30.1 

6.6 

19.8 

5.1 

20.3 

4.9 

25.1 

4.0 

28.1 

14.2 

26.0 

' ■' 5:2 

27.2 

11.0 

25.5 

3.9 




Ouly 11, 1977 





-2.2 

IB 

-7.6 

n 

-8.7 

10 

-30.7 

12 

-23.2 

5 

-12.5 

9 

-1.9 

15 

9.8 

9 

0.8 

5 

-10.1 


Value 

of 

test 

statistic 


-1.7 

-7.8 

-16.6 

16.1 

-16.3 

-5.1 

-31.3 

3.9 

-23.2 

2.7 

-17.9 

7.6 

3.9 

-8.3 

23.1 

26.5 













































































TABLE 5-1 Continued, 
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TABLE 5-1.— Continued 



5-6 









































































































TABLE 5-1.— Concluded, 



• Yield 

‘'Relative 

difference 

Value 

of 

test 

Region 

^USDA/SRS 

LAC IE 


Estimate 

(bu/ac) 

Estimate ‘ 
(bu/ac) 

CV 

{%) 

1976 CV 

i%) 

1977 

(%) 

1976 

statistic 


October 11, 1977 


WINTER WHEAT 

Colorado 

Kansas^ 

Nebraska 

Oklahoma 

Texas 


^USSGP 


Montana 
S. Dakota 


®MW states 


Wp-7 


SPRING WHEAT 

Minnesota 
N. Dakota 


^SW states • 


Montana 
S. Dakota 


MW states 


TOTAL WHEAT 

Montana 
S. Dakota 


MW states 


USNGP 


23.0 

22.5 . 

28.5 

28.8 

35.0 

31.9 

27.0 

• 20.0 

25.0 

. 20,3 

27.8 

25.4 


26.5 

BH 

■ 27.1 

27.8 

26; 6 

27.8 ■ 

25.5 

38.9 

32,0 . 

24.1 

23.0 

27.9 

24.8 

23.2 , 

18.0 

24.0 

' 20.8 

23.6 

19.3 

26.7 

23.4 

■ 25.9 

23.2 

24.7 ' 

22.9 

25.4 

23.1 

■ 26.9 . 

24,1 

27.5 

24.9 . 



4 

-0.4 

-1 

.68 
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LEGEND 
— LACIE 

USD A/S RS 

W = Winter wheat 
S = Spring wheat 
T ~ Total wheat 


Figure 5-1.— LACIE and USDA/SRS yield estimates (bushels/acre). [USDA/SRS 
yield estimates through April 22 derived from predicted production and 
seeded acres estimates released on December 22, 1976.] 


5-S 









yield may vary around the trend curve according to the weather. The three 
Texas models show that the yields do fall above the CCEA trend for 1977. This 
indicates that the weather has been good, yet the LAC IE estimate is still 
approximately 20 percent below the USDA/SRS estimate. In addition to the 
trend factor the May precipitation variable has demonstrated inconsistency in 
the models. The fact that above-normal rainfall detracted from yield in the 
Texas low plains and Oklahoma models, but added to yield in the panhandle 
model, may indicate that the models are not reflecting plant response to the 
full range of weather over the two states. 

For the mixed wheat states, .the difference between the LACIE and USDA/SRS 
estimates of winter wheat yield was not significant in every month for which 
statistics were available. 

The LACIE estimates of yield for the four USNGP spring wheat producing states 
were consistently below their USDA/SRS -counterparts, although the relative 
difference decreased in magnitude in each successive report. The difference 
between the LACIE and USDA/SRS yield estimates for the USNGP region was sig- . 
nif leant at the 10-percent level for each of the three months in which LACIE 
estimates were available (August, September, and October), although that of 
the October estimate was only marginally significant. All state-level LACIE 
spring wheat yield estimates were below their USDA/SRS counterparts. 

The LACIE total wheat yield estimates for the USGP region were below the cor- 
\ 

responding USDA/SRS estimates in August, September, and October. Total wheat 
yield statistics were not available in August and September CMR's. The dif- 
ference between the LACIE and USDA/SRS October total wheat yield estimates 
for the USGP region was significant at the lO-percent level. The differences 
between LACIE and SRS yield estimates in Oklahoma, Texas, Minnesota, and 
Montana were the principal causes for LACIE' s failure to meet the 
90/90 criteria. 


5-9 



5.2 CROP CALENDAR MODEL ACCURACY 

Crop growth stage estimation based on current year weather conditions serves 
two vital components of LACIE: CAMS and the Yield Estimation Subsystem (YES). 

Initially, CAMS relies on the crop growth information early in the year to 
determine whether the wheat is sufficiently emerged to be detectable. Once 
the Robertson model predicts the crop to have emerged (Robertson stage 2.0) 
analysis of the segment for wheat percentage is initiated. The winter wheat 
crop is monitored also to ascertain if it has emerged from dormancy. In some 
northerly regions of the winter wheat producing states of the USGP, crop esti- 
mates are not attempted during dormancy because the canopy 1s too sparse. The 
next major growth period of interest to CAMS is the period after dormancy to 
heading, when the analyst relies on the Robertson crop stage to ascertain the 
approximate expected intensity of the wheat vegetation signature in compari- 
son to other spring-planted crops. Heading to senescence or maturity is 
another key stage in the separation of wheat from other vegetation. During 
this stage, the appearance of the wheat is significantly different from other 
vegetation types. Senescence to harvest and postharvest are very important 
to the analyst because the Landsat acquisitions during this period permit him 
to verify his early-season identifications of wheat. (Wheat, other small 
grains, and grasses mature and are harvested during this period.) 

This very general description of the crop calendar function in CAMS aids in 
qualitatively understanding the effect of growth stage prediction errors. 

For example, if the Robertson model predicts full emergence at a date ear- 
lier than crops are fully emerged (growth model is ahead of actual progress), 
CAMS will analyze the segment in a period when some amount (depending on the 
magnitude of *the growth model prediction error) of the wheat is incompletely 
-emerged. Since incompletely emerged wheat fields will go undetected by the 
analyst, the growth model prediction error can result in a negative bias in 
the segment proportion estimate. In all cases, if the model predictions run 
too far ahead of the actual growth stage, the analyst will anticipate an onset 
of changing signatures within the segment, which will not occur at the pre- 
dicted rate. Thus, if the growth model predicts 90- percent senescence within 
the segment and the analyst bases his labeling decision on this fact, certain 
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fields could be discarded as being nonwheat because a senescent signature was 
expected and the analyst did not observe a change. 

Inasmuch as the interactions between the growth model prediction errors and 
CAMS errors are not fully understood and their relationships to each other 
remain unquantified, substantial prediction errors in the model could result 
in substantial errors in analyst 'label ing. 

The currently implemented operational yield models in LACIE do not depend on 
the crop growth model. However, the response of wheat yield to meteorological 
conditions is known to depend quite strongly on the growth stage at which 
these conditions are present. For example, high temperatures after wheat 
maturity do not affect yields in the same way they do during heading. The 
second-generation yield models being evaluated for LACIE in Phase III depend 
on the crop growth models; the effects of certain meteorologically related 
variables are weighted differently, depending on the estimated growth stage 
of the plant. Thus, errors in. the growth model can strongly influence the 
yield estimation error; e.g., if high temperatures are experienced the last 2 
weeks in May in an area where heading is occurring and the growth model (run- 
ning fast) is predicting that the crop is ripe, the second-generation yield 
models will fail to predict the actual reduction in yield. 

As stated, the relationship between the growth model prediction errors and 
the yield estimation errors is not completely unjderstood, and the effects 
have not been quantified. 

The accuracy assessment effort within LACIE has designed an evaluation of the 
crop growth models, utilizing ground-acquired information from' intensive test 
sites (ITS's) in the yardstick region. This evaluation was conducted over 
eight winter wheat ITS's in Kansas and Texas during Phase^ II and was expanded 
in Phase III to include 22 ITS's throughout .the United States and 11 ITS's 
in Canada (figures 5-2 and 5-3). 


5-n 




Figure 5-2.-- Map of U.S. wheat-producing areas showing 

intensive test sites. 
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Figure • 5-3.— Map of Canada showing intensive test sites. 



Within each of these ITS's, the average ground-observed growth stage for the 
wheat crop is calculated from periodic field-by-field observations obtained 
by personnel from the USDA Agricultural Stabilization and Conservation Ser- 
vice (ASCS). ASCS personnel record detailed information regarding each field 
on the form shown in figure 5-4. The observer specifies the growth stage of 
each field to be one of the 10 stages listed on this form. All sites are 
visited each 18 days by ASCS field personnel, except for the Finney County, 
Kansas, and Hand County, South Dakota "supersites, " which are visited every 9 
days. The 11 ITS's in Canada are monitored each 18 days by personnel from the 
Canadian Agriculture Department. 

The crop calendar model used by LACIE is a modification of the Biometeorolog- 
ical Time Scale (BMTS) developed by Robertson. The Robertson BMTS estimates 
the stages for the progress of wheat crop development from planting to harvest 
(table 5-2). Daily maximum and minimum temperatures and day length are vari- 
ables used to implement this model, which is often referred to as the Adjus- 
table Crop Calendar (ACC), 

All of the growth stages defined by Robertson in the BMTS model development 
are not easily observable by field personnel. For example, BMTS stage 3.0 
(jointing) can be obs'?rved only by plant dissection. Thus, a different set 
of stages has been developed for ground observations. The ground-observed 
growth stage of each ITS must be developed by relating the ITS growth- stage 
observations to the related BMTS .stage. After planting, the earliest stage 
at which there is no ambiguity in this relationship is at heading. The BMTS 
stage 3.0 (jointing) is known to occur after tillering and before booting, 
which are observable by ground personnel. Thus, jointing is estimated by 
extrapolating between these observations. An error as large as a few days is 
customary in relating ground observations to BMTS stages. It should be kept 
in mind that heading is the most valid comparison as the results of the ACC 
are reviewed. 

The ACC is published biweekly in a meteorological summary for all regions 
being ex^minpri hv LACIE. The BMTS stages of wheat are based on inputs from 
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TABLE 5-2.- ROBERTSON BMTS AND OBSERVED ITS WHEAT 
PHEN0L06ICAL STAGES 


Stage 

Robertson 

BMTS 

ITS growth 
stage code 

Description 

Planted 

r.o 

01 

Planted 



02 

Planted, no emergence 

Emergence 

2.0 

03 

Emergence 

Jointing 

3.0 

04 

Tillering, prebooting, pre- 
budding 


3.5 

05 

Booted or budded 

Heading 

4.0 

06 

Beginning to head or flower 


4.5 

07 

Fully headed or flowered 

Soft dough 

5.0 

08 

Beginning to ripen 

Ripening 

. 6.0 

09 

Ripe to mature 

Harvest 

7.0 

10 

Harvest 


each reporting meteorological station. These estimates are then utilized to 
develop BMTS contours as shown in figure 5-5. The ITS BMTS estimate is then 
determined from its location on this contour map and compared to that deter- 
mined by ground observations. Such a comparison is shown in figures 5-6 and 
5-7 for two ITS's. The standard deviation (±la) of these ground-observed 
estimates on a field-to-field basis is also shown in these figures. Note in 
the Oldham County, Te;;as, example that the ground-computed stage contains the 
ACC-estimated stage within one standard deviation in the periods from mid- 
jointing (3.5) to soft dough (5.0). Before 3.5 and after 5.0, the ACC was 
ahead of the ground truth by a few days and more than one standard deviation. 
However, in most cases, the ACC BMTS estimate was somewhat more accurate than 
assuming a normal or average growth stage. In Finney County, Kansas, the 
historic data indicated approximately as well as the BMTS, and both were 
relatively close to the ground-observed information. 
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. GROUND TRUTH PERIODIC OBSERVATION EURH 

oiMlfoS S§ MSHINCTCNI 


WHEAT 
ES/PASTURE 


LAND USE CODES 

^100-SPRlNG WHEAT 
200" BARLEY 
300-QATS 
<^00-WINT 

500-cAaS 

60CH0THER CROPS 
601-RAPESEE0 
6o5-RYE 
604-FLAX 
607-CORN 
617-SOYBEANS 
61B-C0TT0N 
700-SUMHER FALLOW 


GHOMTH STAGES 


03-eM£R0 

0^-TlLUl 


- NO EMERGENCE 
NCE 

INGf PRE0CCT, 
PR^BUD 

05- B00TE0 DR 6UDDEC 

06- BfcGINNING TC HEAD 

OR FLOWER 

07- FULLY HEADED CR 

floherlo 

08- BEGINNING TC RIPEN 

09- PlPE MATURE 

10- HARVESTEO 


GROUND 
CCVER (A| 

L- O- 19 

2- 20- 39 

3- -<»0- 59 

4- 60- 79 

5- 8C-IC0 


SURFACE MaiSTOKE 
CONDITIONS 

1- DRY 

2- QAMP 

3- wET 

4- StANClNG WATER 


HbEO GRCwTH 

1- NEGL IGlbLb 

2- SHGHT 

3- MGUERATfc 

4- HEAVV 


BONXH 

LANOSAT PASS DATE 
OBSERVATION DATE I C / J *2 

RAINFALL SINCE LAST OblfERVATToh 

field operations 


DAY 

Z.vi /7T 
■ ^ /7T 


IN. 


01- bARb GROUND 

02- bARc CISKED/CULTIVATEO 

03- bARfc PLOWED 

04- eARESE£0EU 

05- STANDING STUBBLE 
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ORQHTH/VIELD 

OETRACTANTS 


STAND 

DUALITY 


01- SALINITY l-POOR 
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04- DROUGHT 3-AVERAGE 

05- HOI STURE 4-ABGVE 
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10- POT HOLES ii-HlNT£RKILL 
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900-UNKNOHN CROPS ll-DOES NOT APPLY 14-CTHcR 12-WEEDS 15-OTHER 


FIELD ACREAGE LAND USE 


cn 

I 


o> 


S o 

wS 

ss 

is 

a 


NO. 


43 233.7 


20 511.6 


31 455.0 


30 306.6 


29 401.0 


27 
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39.1 


25 377.1 
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Figure 5-4.— ASCS Ground Truth Periodic Observation form. 
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CRD 11, TEXAS, Wlh 



Figure 5-6.— Comparison of observed and 

Oldham County 





WHEAT DEVELOPMENT STAGE 



CRD 30, KANSAS, WINTER WHEAT, 



Apr. May June July 


Figure 5-7.— Comparison of observed and predicted crop calendar stages for 

Finney County, Kansas. 
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Tables 5-3, 5-4, and 5-5 display the differences in days at which each of the 
BMTS stages was estimated by ground observations and by the LACIE ACC. At 
heading, the standard deviation of the ground observations is about 6 to 9 
days. A difference between the ground-observed and ACC estimates larger than 
±la occurred in only three of the ITS's in the U.S. While statistical anal- 
yses of these data have not been concluded at this writing, it would appear 
that the computed differences between the ground-observed and ACC-estimated 
BMTS stages are not significant in terms of the experimental error. However, 
some trends were noted. In the winter wheat region, the ACC was consistently 
ahead of the ground observations at BMTS stages 5.0 and 6.0 (soft dough and 
ripening) and at jointing. 

While these results do not conclusively demonstrate crop calendar inadequa- 
cies, several issues must be addressed before the ACC technology can be con- 
sidered adequate. For CAMS, the analyst must know, early in the season, the 
expected spectral appearance of the wheat canopy. This signature, however, 
is related not only to the wheat growth stage but also to other factors; e.g., 
whether the field is irrigated and if it was fallowed the previous year, and 
the soil color. Thus, a signature model incorporating the.. ACC. parameter as 
input would be a more desirable product from the analyst’s point of view. 
Another major issue to be addressed is understanding just how crop calendar 
errors affect labeling accuracy. As mentioned at the beginning of this sec- 
tion, these effects are only qualitatively understood at present. 

Whatever the ACC model requirements, the model can be improved for winter 
wheat by developing an additional model to predict the actual planting date. 
Currently, the LACIE ACC is "jtarted" (i.e., the clock is set to 1.0 and 
meteorological data are fed to the model) on a date determined to be the his-' 
torical average planting date for the Crop Reporting District (CRD) in which 
the segment is situated. Since this average planting date can vary consider- 
ably from one year to the ndxt, a sizable error can be introduced into 
growth stage estimation before dormancy for winter wheat. In tests where the 
ACC has been "started" based on the ground-observed planting date, the ACC 
BMTS estimates have been more accurate prior to dormancy. 
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TABLE L-3.“ COMPARISON OF LACIE ACC WITH OBSERVED 
STAGES IN THE WINTER WHEAT ITS'S 

[Monitoring ACC data (in days) between 
TTS and ACC development stages] 


ITS, county/state 

Jointing 

Headi ng 

Soft 

dough 

Ripening 

3.0 

3.5 

4.0 

4.5 

5.0 

6.0 

Randall /Tex. 

3 

7 

5 

4 

8 

8 

Deaf Smith/Tex. 

(a) 

(a) 

(a) 

(a) 

(a) 

(a) 

Oldham/Tex. 

-4 

17 

17 

9 

9 

8 

Finney/ Kans, 

'4 

5 

-3 

3 

8 

-5 

Rice/Kans. 

-12 

0 

-5 

-14 

0 

7 

Ellis/Kans. 

-11 

-3 

-8 

-15 

1 

-11 

Saline/ Kans. 

4 

0 

-3 

-3 

6 

11 

Morton/ Kans. 

2 

0 


0 

5 

8 

Shelby/Ind. 

10 

-1 


. -1 

-4 

2 

Madison/Ind. 

10 

6 

1 

0 

8 

5 

Boone/Ind. 

10 

9 


0 

2 

5 

Oneida/ Idaho 

-11 

-7 


-7 

-5 

(a) 

Frankl in/Idaho 

(a) 

(a) 

(a) 

1 

1 4 

(a) 

Bannock/ Idaho 

15 

3 

0 

-1 

8 

(a) 

Whitman (1)/Wash. 

(a) 

(a) , 

(a) 

(a) 

(a) 

(a) 

Whitman (2)/Wash. 

-5 

10 

-3 

-9 

2 

7 

Whitman (3)/Wash. ■ 

(a) 

(a) 

(a) 

(a) 

(a) 

(a) 

Hi 11 /Mont. ' ■ 

3 

-8 

-9 


5 

(a) 

Liberty/Mont. 

(b) 

(b) 

(b) 

(b) 

(b) 

(b) 

Hand (1)/S. Dak. 

mm 

5 

-5 

0 

(a) 

(a) 

■Hand (2)/S. Dak. 

D 

(a) 

(a) 

(a) 

(a) 

(a) 

Toole/Mont. ■ 

WM 

-8 

'-6 

-9 

-8 

(a) 


^No data. 

^No winter wheat. 
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TABLE 5-4.- COMPARISON OF LACIE ACC WITH OBSERVED 
STAGES IN THE- SPRING WHEAT ITS'S 


ITS, county/state 

Jointing 

Heading 


Ripening 

3.0 

3.5 

4.0 

4.5 

5.0 

6.0 

Hand (1)/S. Dak. 

-10 

-5 

-2 

-8 

(a) 

(a) 

Hand (2)/S. Dak. 

(a) 

(a) 

(a) 

(a) 

(a) 

(a) 

Burke/N. Dak. 

(a) 

(a) 

(a) 

(a) 

(a) 

(a) 

Div1de/N. Dak. 

(a) 

(a) 

(a) 

(a) 

■{a) 

(a) 

Williams/N. Dak. 

(a) 

5 

2 

4 

■■ 

(a) ' 

Hill /Mont. 

10 

12 

6 

6 

HB 

(a) 

Liberty/Mont. 

-19 

(a) 

(a) 

(a) 


(a) 

Toole/Mont. 

2 

(a) 

-1 

6 


(a) 

West Polk/Mont. 

-7 

-5 

-2 

. 6 


(a) 


^No data. 
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TABLE 5-5-- COMPARISON OF LACIE ACC WITH OBSERVED 
STAGES IN THE CANADIAN IIS’S 


ITS, town/ province 

Jointing 

Headi ng 

wm 

Ripening 

3.0 

3.5 

4.0 

4.5 

5.0 

6.0 

Ft. Sask./Alta. 

• -1 

0 

-7 

(a) 

(a) 

(a) 

Olds/Al ta. 

10 

7 

(a) 

(a) 

(a) 

(a) 

Lethbri dge/Al ta-. 

12 

13 

10 

(a) 

(a) 

(a) 

Melfort/Sask. 

• 9 

9 

7 

(a) 

(a) 

(a) 

Delisle/Sask. 

11 

5 


(a) 

(a) 

(a) 

Swift Current/Sask. 

9 

5 

-4 

(a) 

(a) 

(a) 

Torquay/Sask. 

7 

3 

-2 

(a) 

(a) 

(a) 

Stony Ht./Man. 


3 

1 

2 

(a) 

(a) 

Starbuck/Man. 


0 

-3 

-3 

(a) 

(a) 

Altona/Man. 


-1 

-8 

-9 

(a) 

(a) 

Dawson Creek/B.C. 


(a) 

(a) ■ 

(a) : 

(a) 

(a) 


^No data. 
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6- ACCURACY ASSESSMENT SPECIAL STUDIES 

This section presents the results of special studies which were done by 
various AA groups during the LACIE Phase III. These special studies include 
(1) dot labeling errors for ITS's, (2) effect of analyst, acquisition history 
and bias correction, (3) investigations of the winter wheat area overestimation 
problem in South Dakota, (4) comparison of ratioed and direct wheat aggrega- 
tions for North Dakota, and (5) effect of the objective thresholding procedure. 

6.1 ITS STUDY OF DOT LABELING ERRORS 

By mid-September of 1977 there were 108 acquisitions from 16 ITS's in the USGP 
states. Reliable classifications and ground truth data were available to per- 
mit the tabulation and grouping of labeling errors for type-2 dots (bias cor- 
rection dots) from the latest classification of 13 of these ITS's. 

In table 6-1 the errors were divided into errors of omission, errors of com- 
mission, and errors associated with border/edge pixels. The errors of omis- 
sion and commission were subdivided into two categories according to whether 
the pixel did or did not follow the normal wheat development sequence. 

The errors of omission where the pixel did follow the normal wheat development 
sequence were then further subdivided in accordance with wheat development in 
the- ITS, i.e,, whether the development of most of the wheat fields in the ITS 
was in accordance with, behind, or ahead of the development expected from the 
Robertson biostage as determined from the ACC. 

The errors of commission where the pixel did follow the normal wheat develop- 
ment sequence were subdivided into nonwheat and volunteer wheat pixels. The 
■identification of volunteer wheat as wheat is considered to be an error 
because it is not usually harvested. 

A total of 978 type-2 dots .were labeled by analysts for the 13 ITS's and 94 
(9.6 percent) were incorrectly labeled. Errors of omission (64 of 326, or 
19.6 percent) exceeded errors of commission (30 of 652, or 4.6 percent). 
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TABLE 6--1 ERRORS OF OMISSION AND COMMISSION 



Errors of omission 

Errors of commission 







Old the dot follow the normal wheat development sequence? 

Border 

edge 

vol 




i Reason 

Yes 



Yes 




Errors 



Normal® 

development 

Developed 

late^ 

Developed 

early® 

No 

But not 
wheat 

Volunteer 

wheat 

No 




Total 

Total 

dots 


^ No. 

% 

No. 

% 

No. 

* 

No. 


No. 

% 

No. 

% 

No. 

% 

No. 

% 

No. 

% 

1 

Necessary acquisitions 
not available 









6 

6.4 



. 




6 

6.4 

0.6 

Poor stand of wheat 



3 

3.2 





1 

1.1 







4 

4.3 

0.4 

Late planting! emergence, 
or development 

10 

10.6 

24 

25.5 









2 

2.1 



36 

38.3 

3.7 

Difficulties caused by 
narrow strip fields 





Z 

2.1 











2 

2.1 

0.2 

Clerical error^ 

13 

13.8 











4 

4.3 



17 

18.1 ' 

, ! 

1.7 

Confused with other crops 

9 

9.6 







2 

2.1 

4 

4.3 

1 

! 

1.1 



16 

17.0 

1.6 

Border edge pixel 













3 

3.2 



3 

3.2 

0.3 

Unknown 

i 


1 

1.1 

2 

2.1 







6 

6.4 

1 

1.1 

10 

10.6 

1.0 

Total 

32 

34.0 

28 

29.8 

4 

4.3 



9 

9.6 

4 

4.3 

16 

17.0 

1 

1.1 

94 

100.0 

9.6 


^Development refers to the development of most of the wheat fields in the ITS relative to the Robertson biostage of the adjustable 
crop calendar, 

^Clerical error: 

1. Wrong acquisition requested for classification. Analyst simply wrote the wrong number Inadvertently. 

2. Pixel misidentlfied by mistake. Same signature on other pixels was consistently identified as nonwheat. 



The largest single cause of labeling errors was late planting* emergence, or 
development of small grains. This category of error accounted for 36 
{.38.3 percent) of the total errors. Other major causes of errors were anal- 
yst errors (17 errors, -or 18.1 percent), and confusion with other crops (16 
errors, dr 17.0 percent). 

Most errors were categorized with approximately equal weight given to each of 
the following reasons; 

a. Omission errors because the signatures were unusual in an otherwise normal 
temporal development pattern . 

b. Omission errors because the wheat fields were well behind the normal 
development pattern according to the ACC. 

c. Commission errors because the wheat signature was confused with other 
signatures for a variety of reasons, mainly confusion with native vegeta- 
tion or hay. 

Similarly, the three greatest causes of error were: late emergence, inconsis- 

tencies in labeling, and confusion with other crops. 

Table 6-2 shows the types and causes of labeling errors for each of the 13 
ITS's. Only 4 of the 13 ITS's had more than 10 labeling errors and in each 
one many of the errors had the same cause. In the Hand County #2 segment, all 
eight errors of omission were normally developed wheat which was confused 
with native pasture. In Morton County all 13 omission errors were late 
emerged or developed wheat which was confused with hay. Ten of the errors 
of omission in Rice County were normally developed wheat with an unusual color 
signature (such as purple, magenta, or brown). Six of the errors of commission 
in Randall County were nonwheat pixels which appeared to be wheat on available 
acquisitions. (Additional acquisitions were needed.) 

6.2 EFFECTS OF ANALY3T INTERPRETER (AI), 'ACQUISITION HISTORY, AND BIAS 
CORRECTION ON PROPORTION ESTIMATION ERROR 

The Image 100 processor and data from eight U.S. blind sites were used in an 
experiment wherein each site was analyzed fay three AI's to give a procedure 1 
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TABLE 6-2.- LABELING OF ERRORS AT SEGMENT LEVEL 


Segment 

Robertson biostage 

LACIE biowindow 

True wheat, % 

Wheat dots 

Other dots 

No. Incorrect labels 

Incorrect labels, % 

Necessary acquisitions 
not available 

Poor stand of 
wheat 

Late planting, emergence, 
or development 

Difficulties caused by 
narrow strip fields 

Clerical errors^ 

Confused with other 
crops 

Border/edge pixels 

Unknown 

No. 

Incorrect 

labels 

No. 

Incorrect 

labels 

.. . . _ 1 

No. 

% 

No. 

% 

1687 Hand #1, SO 

2 

1 

15.9 

26 

2 

IJ 

72 

8 

11.1 

10 

10.2 





4 

4 


Z 

1986 Hand #2, SD 

^4-3 

®3-2 

1.3 

9 

6 

88.9 

89 

3 

3.4 

11 

11,2 






10 


1 

1961 Morton, KS 

3 

2 

42.8 

35 

13 

37.1 

64 

0 

0 

13 

13.1 



13 






1962 Saline, KS 

5 

3 

62.9 

36 

1 

2.8 

24 

4 

16.7 

5 

8,3 





1 


3 

1 

1963 Rice, KS 

6 


35.0 

23 

11 

47.8 

37 

0 

0 

11 

18.3 



10 



1 



1964 Elvis, KS 

5 

3 

48.3 

23 

10 

43.5 

35 

0 

B 

10 

17.2 



•3 

2 

3 



2 

1988 Finney, KS 

3 

2 

42.9 

21 

4 

19.0 

31 

1 

3.2 

5 

9.6 


3 

— 


2 



m 

1970 Liberty, MT 

B 

1 

3.3 

8 

0 

0 

91 

2 

2.2 

2 

2.0 



2 





B 

1971 Hill, HT 

B 

3 

32.1 

33 

6 

18.2 

57 

1 

1.8 

7 

7.8 

B 

B 

3 


3 

1 


B 

1978 Randall, TX 

5 

3 

43.5 

23 

3 

13.0 

30 


30.0 


22.6 



2 

B 

B 




1979 Deaf Smith, TX 

2 

1 

27.7 

20 

2 : 

10.0 

78 

n 

1.3 


3.1 



2 

B 

IBB||| 



1 


6 

4 

3Q.0 

31 

4 ; 

12.9 

22 

D 



Q| 

■n 

1 

1 


3 




1987 West Polk. MN 

B 

3 

60.1 

38 

0 

0 

22 


0 

0 





B 

B 

■H 



Total 

t 




326 

64 

19.6 

652 

30 

4.6 

94 


6 

B 

36 



Lli. 

3 

10 


^M1xed wheat. 

^Clerical error: 

1. Wrong acquisition requested for classification. Analyst simply wrote the wrong number inadvertently. 

2. Pixel misidentified by mistake. Same signature on other pixels was consistently identified as nonwheat.- 



















































































































"raw" and a "bias-corrected" estimate of the proportion of small grains in each 
segment. The segments were of two types; namely, those having acquisitions in 
all four biophases and those having only early season acquisitions. The seg- 
ments were chosen at random from the blind sites for which detailed ground 
truth was available. 

■The objectives of the experiment were: (1) to evaluate the performance of 

procedure 1 in terms of absolute proportion estimation' error and its repea ti- 
bility over AI's, (2) to make comparisons between "bias-corrected" and "raw" 
procedure 1 estimates, and (3) to determine if the performance was better when 
acquisitions from all biostages were used than when only the early season 
acquisition was used. 

The. third objective could not be achieved properly because of the small number 
of segments used (four of each type). It was later estimated that to make 
effective comparisons of this type in a fully nested design, one would need 
about 10 times as many segments. The efficiency of the test could be improved 
if the same segments were analyzed first using only early-season acquisitions 
and then using all acquisitions; however, there would be potential biasing 
problems in such replication if the same AI analyzed the segment under both 
the early-season and full-season conditions. If different AI's performed the 
analysis, the resulting large variability would destroy the power of the test 
just as the large segment variability destroyed it in the experiment reported 
here. 

Table 6-3 shows the absolute proportion estimation error |X - X] where X is 

/\ 

the ground truth small grains proportion and X is the analyst's estimate of X 
for the various treatment combinations. Averages are blocked off from the 
basic data; for example, the average absolute error for AI "B" on early-season 
segments was 11.6 for the raw estimate and 11.8 for the bias-corrected esti- 
mate. The average absolute error on all segments was 7.9 for raw estimates 
and 11,1 for bias-corrected estimates. The average absolute error for all 
three AI's was 12.8 for raw early-season estimates, 6.3 for raw full -season 
estimates, and 9.5 for all eight segments with the raw estimate. The grand 
mean was 10.0. 
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TABLE 6-3.- IMAGE lOO - PROCEDURE 1 DATA 
[|X r X| (pall grains)] 


Acquisition 

history 

Raw 

Bias correction 

Overall 

average 

Analyst 

Average 

Analyst 


A 

B 

C 

A 

B 

C 

Early 

season 

only 

Average 

16.5 

10.8. 

2.0 


18.9 

■ 8.7 

16.8 

* 


11.4 

Ts.s 

21.3 

5.6 

18.3 

19.7 

ED 

14.6 

mjj 


11.9 

19.5 

8.4 

2.5 

EO 

1.6. 

8.2 

1.5 

11.5 

11.6 

15.2 

12,8' 

00 

11.8 

14.4 

11.6 

12.2 

Full 

season 

Average 

0.8 

1.4 

0.9 

.. ^ 

1.4 

1.4 

2.0 


' 

5.2 

10.6 

31.7 

9.7 

32.9 

32.6 

1.3 

BB 

15.1 

7.2 

5.0 

14.0 

1.7 

ID 


2.7 

2.5 

2.5 

2.2 

4.3 

12.5 

6.3 

5.3 

10.5 

12.8 

9.5 

-7.9 

Overall 

average 

6.9 

7.9 

13.8 

9.5 

6.9- 

11.1 

13.6 

10.5 

10.0 
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The most obvious features of table 6-3 are the large variability between AI's 
and between segments. If this variation is taken to be typical, then future 
experiments should be-designed so that segments and AI's are "crossed" with 
treatments as much a? possible. 

Analysis of variance- was used, to test for the effects of AI's, time (i.e., 
early-season versus all acquisitions), method (raw versus bias correction), 
and their interactions. The results are shown in table 6-4. They lead to 
the following conclusions: 

a. The large disparity between data from various AI's was not consistent 
over segments; i.e., an AI would do better on one segment than on another 
one. 

b. There was no significant difference between methods; i.e. the use of bias 
correction just traded one random error for another one of comparable 
magnitude, 

c. Any test involving "times" 'was not significant- As stated earlier, these 
tests had extremely low power because of insufficient numbers of segments. 

6.3 INVESTIGATIONS GF THE WINTER WHEAT AREA OVERESTIMATION PROBLEM IN 
SOUTH DAKOTA 

The LACIE winter wheat area estimate for July was below the USDA/SRS estimate 
at the USSGP level and above the USDA/SRS estimates for the mixed wheat states 
and the USGP. This was primarily because of the large overestimate in South 
Dakota as shown in table 6-5. A relative difference of 85.3 percent was 
reported in South Dakota, indicating a large overestimate. In fact, the 
LACIE area estimate for South Dakota was approximately seven times greater 
than the corresponding USDA/SRS estimate. An investigation of this problem 
was conducted by various elements of LACIE. 

Several factors contributed to the overestimation of winter wheat area in 
South Dakota. One factor was the CAMS overestimation of winter wheat area in 
marginal wheat areas. A second factor was the sensitivity of the aggregation 
model to overestimation in such areas. Also, it is possible that part of the 
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TABLE 6-4.- ANALYSIS OF VARIANCE 


Source . 

Degree of 
freedom 

SS® 

MS** 

F*' 

T (TIME) 

1 

215.48 

215.48 

0 . 70 ’' 

M (METH) 

1 

11.50 

11.51 

0.41^ 

TM (TM) 

1 

56.55 

56.55 

2.01^ 

A (AI) 

Z 

379.05 

■ 189.52 

2.53 

AT 

Z ‘ 

34.51 

17.25 ■ 

0.23 

AM 

2 

29.91 

14.71 

0.68 

TAM 

2 

16.50 

8.25 

0.38 

S/T 

6 

1840.09 

306.68 

4.09* 

MS/T 

6 

168.94 

28.16 

1.30 

- AS/T 

12 

899.63 

74.97 

3.46^* 

MAS/T 

12 

259.65 

21.64 



^SS - Sum of Squares. 

^MS - Mean square. 

- F-value. 

LEGEND: 

p 

1 - Assume 0 AT = 0 

2 - Assume g^AM = 0 
*3 - Assume a^TAM = 0 
*P < .025 

X - Conservative test (inflated denominator) 
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TABLE 6-5.- COMPARISON OF SOUTH DAKOTA AND MONTANA WINTER WHEAT ESTIMATES 
USING REDESIGNATED SEGMENTS WITH USDA/SRS AND LACIE ESTIMATES 



No. segments 
(acq/alloc) 

Area _ 
(ac lO"^) 

$.E.% 
(ac X lO"^) 

CV 

{%) 

RD^ 

{%) 

WSBM 

SOUTH DAKOTA 

• 






• % 
USDA/SRS 


680 





LACIE 

39/56 

4629 

583 

•12.6 

85.3 

3991.8 

Redes. LAClE 

20/26 

1323. 

713 

53,9 

48.6 

960.1 

MONTANA 




- 



USDA/SRS 


2800 





LACIE 

. 58/80 

3097 

380 

12.3 

9.6 

482.3 

Redes. LACIE 

46/58 

2902 

386 

13.3 

3.5 

399.2 


^S.E. = Standard error, 

^RD = Relative difference. 

*^MSE = Variance + (bias)2, where bias is estimated by (LACIE - SRS). 
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overestimation might have been due to the use of CRD-level ratios of winter 
wheat to winter small grains to determine the winter wheat proportion. To 
check this, aggregation using the 1975 USDA/SRS county-level ratios was per- 
formed and a large overestimation still resulted. Use of the CRD ratios 
resulted in an acreage estimate about seven times larger than the USDA/SRS 
estimate, compared to an estimate six times larger using the county ratios. 
This seemed to indicate that overestimation of winter small grains proportions 
by CAMS was the greater problem. 

The aggregation logic in the CAS system is especially sensitive to proportion 
estimation errors in marginal areas. As an example, consider the collection 
of Group II counties in CRD 90 in South Dakota, The epoch year winter wheat 
area data and number of segments allocated to each of these counties were as 
f ol 1 ows : 


iy/4 uensus 


County 

No. segments 
acqui red/al 1 oca ted 

winter wheat area 
(acre x 10^) 


1/1 

21 

'•2 

0/0 

371 

''a 

0/1 

443 

^4 

1/1 

79 


0/0 

0 

% 

1/1 

271 

*-7 

0/0 

939 

'8 

1/1 

375 


TOTAL 4/5 

2 499 


With the four acquired segments, the LACIE 
this Group II collection is given by: 

X r 2499 c ^ 2499 c + 2499 
4 L 21 1 79 4 TTj- 


estimate of winter wheat area for 


C + 2499 
5 375 




= 29.75 + 7.91 C^ + 2.31 Cg = 1.67 Cg 
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where C^. is the estimate of winter wheat area in the Hh county (i=l,...,8) 
as determined from tne winter wheat proportion estimate of the segment in the 
Hh county. Note that even a small overestimation of the C- , particularly C-. 

A * * 

and C4, could lead to a gross overestimate for the collection. In the duly 
CHR, the estimate for the segment in county C-j was 38 times larger than the 
historical county winter wheat proportion with obvious results — extreme 
overestimation for the collection. 

Observing the 1974 area, or production, of winter v^heat for these counties, 
it is obvious that none of them should have received a segment to estimate 
winter wheat area. They did so because a new allocation was performed for 
Phase III based on total small grains. This resulted in the allocation of 
sample segments to areas containing small grains but little or no winter 
wheat in South Dakota. Also, the new allocation required both a winter wheat 
and a spring wheat estimate for segments in areas designated as mixed; i.e., 
containing both winter wheat and spring wheat. Several counties in South 
Dakota, a mixed region, contained a significant portion of spring wheat but 
not winter wheat. This resulted in CAMS having to make winter wheat estimates 
where there was likely to be no winter wheat. 

To avoid the overestimation of area in sparse wheat (winter or spring) regions 
a redesignation of the segments as winter (W), spring (S), or mixed (M) was 
made for the remainder of Phase III. This .had no great impact on the CAS 
aggregation procedures. . No spring wheat estimates had been made for those 
segments designated as pure winter segments and vice versa. The estimates 
for these counties were made using the Group III estimator, or the Group II 
estimator if the county was a Group II collection containing a county for 
which an estimate was made. Of course; both spring wheat and winter wheat 
estimates were made for those segments designated as mixed. 

Currently, action is being taken to redesignate Group III counties in the 
USGP using the epoch year wheat production rather than the small grains pro- 
duction used in the Phase III allocation. This will indicate counties that 
received segments but should not have and' counties that did not receive seg- * 
ments but should have. 
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Aggregations have been performed using the above-mentioned designations, based 
on segment data from the July 11 CMR, to obtain state winter wheat area esti- 
mates for South Dakota and Montana. When bias is a problem, as is the case 
here, the preferred statistic for comparison of two estimators is the mean 
squared error (MSE) or its square root, where the MSE is given by the vari- 
ance of the estimator plus the square of the bias of the estimator. Assuming 
the USDA/SRS estimate to be the true- value, the bias is estimated by the dif- 
ference between the LACIE and the USDA/SRS estimates. The results are pre- 
sented in table 6-5. It is apparent that the redesignation of segments 
improved the LACIE estimate considerably, particularly in South Dakota where 
there is very little winter wheat, according to USDA/SRS data. Although an 
increase in the standard error is noted using redes i gnat ions, the large reduc- 
tion in bias resulted in a 76-percent decrease in the square root of the mean 
squared error for South Dakota.' In Montana, there are many more acres of 
winter wheat than in South Dakota and the use of redesignated segments re- 
sulted in only a slight improvement, a 17-percent reduction in the square 
root of the mean squared error. 

6.4 COMPARISON OF RATIOEP AND DIRECT WHEAT AGGREGATIONS 

For the CMR's of August, September, and October, two types of proportion esti- 
mates were made by CAMS for the segments in North Dakota. First-, as usual , 

CAMS estimated the spring small grains percentage for each segment. These 
estimates were passed to CAS and ratioed down to spring wheat proportions^ 
-before aggregation. In addition, CAMS estimated spring wheat proportions, 
directly for these same segments. These estimates were also aggregated by 
CAS. 

The results of the two aggregations are shown in table 6-6 along with the cor- 
responding USDA/SRS estimates. The CV's for the direct wheat estimates are 
slightly smaller than those for the ratioed wheat estimates in all three months. 
However, the relative differences for August and September are larger (in 
absolute value) for the ratioed wheat estimates. In October the relative dif- 
-ference for the direct wheat estimate was larger. In August both estimates 
were significantly different from the USDA/SRS estimate. In September the 
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TABLE 6-6.“ COMPARISON OF RATIOED AND DIRECT SPRING WHEAT (AGGREGATION) AREA ESTIMATES FOR NORTH DAKOTA 


Month 

of 

estimate 

* USDA/SRS 
area 

estimate . 
(ac X 10^) . 

LACIE 

RD(%) 

Value . 
of test 
statistic 

Estimate 
(ac X 10^) 

CV(%) 

Ratioed 

wheat 

Direct 

wheat 

Ratioed 

wheat 

Di rect 
wheat 

Ratioed 

wheat 

Direct 

w^*eat 

Ratioed 

wheat 

Direct 

wheat 

August 

9530 

6761 

7525 

8.6 

9.6 

-41.0 

-26.6 

-4.8* 

-2.8* 

September 

9530 

8678 

9828 

4.6 

5.2 



-9.8 

3.0 

-2.1* 


October 

9530 

9173 

10604 . 

4.4 

4.8 

-3.9 

10.1 


2.1* 


*Mjj is significantly different from zero at the 10-percent level. 

N . 

Up is not significantly different from zero at the 10-percent level. 




direct wheat estimate was not significantljr different from the USDA/SRS esti- 
mate and in October the ratioed wheat estimate was not significantly different 
from the USDA/SRS estimate. 

A blind site study was performed using the ratioed and direct estimates for 

North Dakota from the October 11 CWR. Figure 6-1 shows plots of the propor- 

/\ 

tion error X - X, where X is the LACIE proportion’ estimate and X is the ground- 
truth proportion obtained using the dot-count ground-truth proportions. 

Table 6-7 shows that the results of the statistical calculations for the same 
data were closer to the dot-count ground-truth proportion estimates than 
were the ratioed estimates. Like the aggregation study, the blind site study 
indicated a higher degree of variability in the direct wheat estimates, as 
evidenced' by the plots shown in figure 6-1. 

In both studies, the October results reveal underestimation in the ratioed 
wheat estimate and overestimation in the direct wheat estimate. 

6.5 EFFECT OF THE OBJECTIVE THRESHOLDING PROCEDURE 

Investigations of the early-season estimates in Phase II disclosed the pres- 
ence of an early-season bias or underestimate of harvestable wheat area. This 
was caused by wheatfields with insufficient canopy development, which were 
not detectable by Landsat. LACIE began Landsat data processing when, the nor- 
mal crop calendar reached biostage 2,0 (emergence) on the Robertson growth 
scale. As the season progressed, ground cover within the fields increased, 
and the LACIE area estimates converged toward the area harvested. Because of 
cloud cover and data drop, some segments were not acquired after complete 
emergence. However, wheat area estimates based on the early acquisitions for 
these segments were utilized to make area estimates throughout the season in 
the conventional aggregations. This contributed to the tendency to under- 
estimate wheat area at harvest. 

In Phase III, an objective thresholding procedure was developed to eliminate 
segments with incomplete emergence from consideration in the overall area 
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October ratioed spring wheat estimates 



X(%) 

Figure b-l.— Plots of proportion estimation errors versus dot-count ground- 
truth proportion estimates for the blind sites in North Dakota. 
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TABLE 6-7.- COMPARISON OF RATIOED AND DIRECT SPRING WHEAT BLIND SITE 
PROPORTION ESTIMATES (EXPRESSED IN PERCENTAGES) FOR NORTH DAKOTA 


Estimate 

n/M 


X 



90% confidence 

X 

D 

% 

interval for Up 

Ratioed 

20/103 

21.0 

25.1 

-4.1 

1.5 

(-6. 7,-1. 5)* • 

Direct ‘ 

20/103 

25.6 

25.1 

0.5 

1.8 

{-2. 6,3. 6)^- 


is significantly different from zero at the 10-percent level, 
is not significantly different from zero at the 10-percent level. 
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estimates.. The thresholding procedure can be applied only at mid-season after 
several opportunities to acquire and estimate wheat percentages have occurred. 
This procedure- was tested in Phase III and was demonstrated to decrease the 
magnitude of the underestimate throughout the season. Therefore, in addition 
to the conventional (nonthresholded) LACIE estimates, CAS also provided the 
thresholded estimates in the June and July CMR's. 

The Application Evaluation System of LACIE established Robertson biostage 2.55, 
as determined from the ACC for crop year 1977, as the wheat detection thres- 
hold of the LACIE system for all winter wheat states except Texas, which uti- 
lizes biostage 2.6, These thresholds were applied to the Landsat data, and 
no segments acquired before the detection threshold were included in the 
thresholded aggregation. 

In table 6-8, the LACIE thresholded and conventional estimates of winter wheat 
area for the seven states and for the regional levels are compared with the 
USDA/SRS estimates. ’ In June, area estimates from all regions and states 
except Nebraska increased after the thresholding procedure was utilized. 
Nebraska showed a slight decrease in the area estimate. These changes in the 
estimates took them closer to agreement with the USDA/SRS estimates in four 
of the seven USGP win-ter wheat states but increased the relative difference 
at the US6P-7- level. This increase was caused by a sampling problem in the 
mixed wheat states. The CV's were increased only slightly by the thresholding 
except in ’South Dakota {where the greatest increase in the estimate occurred) 
and at the USGP-7 level. The CV for South Dakota jumped from 34 to 60 per- 
cent and that for the USGP-7 region went from 5 to 18 percent. This increase 
resulted from the decrease .in the number of segments used for aggregation. 

As shown in the July CMR, area estimates for the five USSGP winter wheat pro- 
ducing states changed only slightly after thresholding-. The CV's for these 
estimates remained constant. The small observed differences between the 
thresholded and conventional estimates resulted from a large number of seg- 
ment acquisitions after emergence and, therefore, minimal thresholding. 

Recorded changes were in the form of mixed increases and decreases among the 
seven states. 
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TABLE 6-8.- COMPARISON OF THRESHOLDED WITH CONVENTIONAL AREA ESTIMATES 


Region 

USDA/SRS 

Thresholded LACIE 

' Nonthresholde 

d LACIE 


Est_ 

(acxlO-^) 

Est, 

(acxlO*) 

n/M 

hI 

cv® 

(%) 

Est, 

CacxlO'^) 

n/M 

RD®' 

cv® 

m 




June 7, 

1977 CMR 





Colorado 

2360 

''3S00 

12/32 

32*6 

15*7 

3065 

1 

28/32 

23.0 

15.8 

Kansas 

12000 

11743 

82/121 


5,5 

10915 

112/121 

-9.9 

S.S 

! Nebraska 

3050 

3603 

22/67 


13.3 

3610 

50/67 

15.5 

12.1 

1 Oklahoma 

6500 

5307 

40/46 


8.0 

4875 

45/46 

-33.5 

9.0 

Texas 

4400 

4910 

26/38 


13.1 

4529 

34/38 

^2.8 

11.9 


28310 

29063 

182/304 

2.6 

4.3 

26994 

269/504 

BB 

4.2 

Montana 

2800 

4188 • 

3/80 

33.1 

28.8 

3253 

41/80 

13.9 

19.2 

S, Dakota 

680 

13759 

S/56 

95.1 

10.8 

2601 

28/56 

73.9 

34.0 

^MW States 

3480 

17947 

8/136 

80.6 

10.7 

5854 

69/136 

40.6 

18.5 

“^USCP-? 

31790 

47010 

190/440 

32.4 

4.9 

32848 

338/440 

3.2 

4,8 




July 

11, 1977 CMR 





Colorado 

2360 

2781 

25/32 

15.1 

15.5 

2962 

30/32 

20.3 

13.2 

Kansas 

12300 

1252 4 

98/121 

1.8 

4.8 

11764 

111/121- 

‘‘ -4^6 

5.0 

Nebraska 

3050 

3746 

34/67 

IS. 6 

11.6 

3475 

52/67 

12,2 

12.4 

Oklahoma 

6500 

5628 

37/46 

-15.5 

7.5 

5264 

42/46 

-23.5 

8.5 

Texas 

4600 

4625 

29/38 

0.5 

12.8 

4511 

34/38 

•-2.0 



28810 

29304 

223/304 

1.7 

■B 


269/304 

-3.0 

Di 

Montana 

2800 

2629 

44/80 

-6.5 

11.9 

3097 

58/80 

9.6 

9 

S. Dakota 

680 ' 

5671 

32/56 

88.0 

15.8 1 

4629 1 

39/56 

85.3 

BB 

^MW States 

3480 

8300 

76/136 . 

58.1 

10.1 1 

7726 

97/136 

55.0 

'9.0 

^SGP-7 1 

! 

32290 

37604 

299/440 

14.1 


3S701 

366/440 

9.6 

3.6 


n * number of segments used. 

H = number of segments allocated. 

^Relative difference - x'loojx. 

^U-S. southern Great Plains region. 


^The mixed wheat states, Montana 
and S. Dakota. 

*^5even -state winter wheat region of 
U.S. Great Plains, 

^Coefficient of variation. 
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In June and July, the winter wheat areas were largely overestimated by both 
procedures. The oversstimation was caused by an inappropriate sampling 
strategy- for the mixed wheat .areas. The detailed results of the investiga- 
tion of this problem are presented in section 6.3. 

Table 6-9" presents the LACIE thresholded and conventional estimates of winter 
wheat yield for the USGP-7 states and for the regional levels in comparison 
with the USDA/SRS estimates. The thresholding procedure should have no effect 
upon the LACIE yield estimates, at the pseudo zone level. The difference be- 
tween conventional and thresholded yield estimates for the state or higher 
levels is due to the different weighting factors for the thresholded and con- 
'v.entional area estimates applied at the pseudo zone level*. The results 
presented in table 6-9 indicate that the thresholding procedure had very 
little -effect upon the LACIE yield estimates at the state and regional levels. 

In table 6-10 the LACIE thresholded and nonthresholded estimates of winter- - 
wheat production for the USGP-7 and the regional levels are compared with the 
USDA/SRS estimates. In. June, thresholding increased all of the production 
estimates at both the state and regional levels. At the USSGP level, this 
increase resulted in an improvement in the relative difference from -23.6 per- 
cent to -15-.0 percent. In Montana and South Dakota, thresholding reduced the 

t 

number of usable segments to 3 and 5, respectively. This is clearly not enough 
segments to make a reliable estimate, as evidenced by very large increases in 
the estimates for these two states, particularly South Dakota, which increased 
by more than a factor of 5 as a result of thresholding. The thresholded esti- 
mate at the USGP-7 level was less accurate than the nonthresholded estimate 
mainly due to the increase in South Dakota. 

In July, estimates increased as a result of thresholding in the three regions 
and in all states except Colorado and Montana. The number of segments thres- 
holded at the USGP level decreased from 148 in June to 67 in July largely as 
a result of a number of later acquisitions becoming available in Montana and 
South Dakota. Thesholding decreased the relative difference at the USSGP 
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TABLE 6-9.- COMPARISON OF THRESHOLDED WITH CONVENTIONAL YIELD ESTIMATES 


Region 


Colorado 

Kansas 

Nebraska 

Oklahoma 

Texas 


‘^USSGP 


Montana 
S. Dakota 


SiW States 


USDA/SRS 

Thresholded LACIE 

Nonthresholded 

Est 

Est 

RD^ 

CV® 

Est 

RD® 

Cbu/ac) 

(bu/ac) 

C%) 


(bu/ac) 

C%) 


June 7, 1977 CMR 


July 11, 1977 CMR 



Colorado 

24.0 

23. ,6 

'-1.7 

16.9 

23.6 

-1.7 

16.9 

Kansas 

33.0 

28.3 

-16.6 

10.6 

28.3 

-16.6 

10.6 

Nebraska 

35. 0 

30.6 

-14.4 

9.8‘ 

30.1 

-16.3 ' 

6.6 

Oklahoma' 

26. 0 

19.8 

-31.3, 

5.1 

19.8 

-31.3 

5-1 . 

Texas 

25. 0 

20.2 

-23.8 

5.0 

20.3 

-23.2 , 

4.9 


29.6 

25.1 

-17.9 

4.0 

25,1 

-17.9 

4.0 

Montana - 

27.0 

28.1 

3.9 

14.2 

28.1 

3.9 

14.2 

S. Dakota 

20.0 

26.0 

23.1 

19.2 

26. 0 

23.1 

19.2 

’"MW States 

25.6 

26.5 

3.4' 

11.3 

27.2 . 

5.9 

11-0 

‘^USGP-7 

29. 2 

25.6 

-14.1 

: 7.8 

25.5 

-14.5 

3.9 


23.0 

22.5 

-2.2 

14.8 

22.5 

-2.2 

31. 0 

28.8 

-7.6 

9.7 

28.8 

,-7.6 

35.0 

32.1 

-9.0 

9.7 

32.2 

-8.7 

26.0 

19.9 

-30.7' 

10.9 

19.9 

-30.7 

25.0 ■ 

20.3 

-23.2 

10-5 

20.3 

-23.2 

28.7 

25.6 

IIQQyi 

* 

25.5 

-12.5 

27.0 

26.5 

-1.9 

12.1 

26.5 

-1.9 

24.0 

26.6 

9,8 

18.9 

26.6 

9.8 

. 26.4 

26. 6 

0,8 

* 

26.6 

0.8 

28.4 

25.8 

-10.1 

5.5 

25.8 

-10.1 


14...8 

9.7 

9.3 

10.7 

10.1 


5.5' 


12.1 

16.9 



^Relative difference = ^ 

^U.S. southern Great Plains region. 

^he mixed wheat states, Montana and S. Dakota. 


Seven-state winter wheat region of 
U.S. Great Plains. 

^Coefficient of variation. 

*Data not available. 
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TABLE 6-10.- COMPARISON OF THRESHOLDED WITH CONVENTIONAL PRODUCTION ESTIMATES 


Region 

'USDA/SRS 

Thresholded 

LAC IE 


Nonthresholdcd 

1.ACIE 


Est. 

(bu*10^) 

(buxlO'^) 

• 

n/M 

RD® 

{») 

cv® 

(4) 

Est, 

(bu*10^) 

n/M 

RD^ 

cv® 

C%) 




June 

7, 1977 CMR 





Colorado 

56640 

82752 

12/32 

31.6 

21,8 

72456 

28/32 

21.8 

21.9 

Kansas 

396000 

33176S 

82/121 

-19.4 

1K6 

308387 

112/121 

-28.4 

IKS 

Nebraslca 

^106750 > 

110081 

22/67 

3.0 

18.7 

108793 

50/67 • 

1.9 

16.2 

Oklahoma 

169000 

104958 

40/46 

-61.0 

13.1 

96S50 

45/46 

-75. 0 

14,0 

Texas 

110000 

99394 

25/38 

-10.-7 

15.9 

91965 

34/38 

-19.6 

14,2 


338390 



-15.0 

m 

mnRun 

269/304 

-23.6 

6.9 

Montana 

75600 

117700' 


35.8 

28.1 

91417 

41/80 

17.3 

23.2 

5. Dakota 

13600 

358030 


96.2 

62. Z 

67685 

28/56 

79.9 

38.3 

^^fW States 

89200 

475730 

8/136 

81.2 

47.3 

159102 

69/136 

43.9 

■Bfl 

'^USGP-7 

. 927590 

1204680 

190/440 

23.0 

19.2 

837254 

388/440 

-10.8 

7,0 




July 

il, 1977 CMR 





Colorado 

54Z80 

62436 

25/32 


21.3 

66516 

30/32 


19.7 

Kansas 

581300 

561294 

98/121 

H 

10.8' 

339343 

111/121 



Nebraska 

106750 

120392 

34/67 

11,3 


111903 

52/67 

4.6 

IS. .7 

Oklahoma 

169000 

112045 

37/46 

-50.8 

13.1 

104907 

42/46 

-61.1 

13.6 

Texas 

115000 

93817 

29/38 

-22.6 

14.-8 

91691 

34/38 

-2S.4 

13.9 


826330 


223/504 

-10.2 

* 

714365 

269/304 

-IS. 7 

* 

Montana 

75600 

69581 

44/80 

-8.7 

16.9 

81983 

58/80 

7.8 

17.2 

S. Dakota ^ 

16320 

150933 

32/56 

89.2 

23.2 

123196 

39/56 

86.8 

22.6 

SlW States 

91920 


76/136 

58.3 

* 

20S179 

97/136 

55.2 

* 

‘HjSGP-7 

918250 

970498 

299/440 

m 

6.6 

919544 

366/440 

m 

6.4 


n = Humber of segnents used. 

M * Number of segments allocated. 

^Relative difference ^ 10o)x. 

^U.S. southern Great Plains region. 


^The mixed wheat states, Montana 
and S. Dakota. 

^Seven- state winter wheat region 
of U.S. Great Plains. 

^Coefficient of variation, 

*Data not available. 
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level and increased it for the mixed wheat states and for the USGP. CV's 
were only slightly changed by thresholding. 
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7. ASSESSMENT OF LACIE ESTIMATES FOR U.S.S.R. 


This section presents an evaluation of LACIE estimates of production, area, 
and yield for the winter, spring, and total wheat crops in the U.S.S.R. The 
reports included in this evaluation are the CMR’s of August 5, September 7, 
and October 5, 1977, LACIE estimates on the U.S.S.R. grain situation are com- 
pared to those-provided by a USDA Interagency Task Force (herein referred to 
as the "Task Force") composed of the Foreign Agricultural Service (FAS), the 
Environmental Research Service (ERS), the Agricultural Stabilization and Con- 
servation Service (ASCS), the Agricultural Marketing Service, and the Office 
of the General Sales' Manager. Although LACIE estimates are provided for each 
region of the U.S.S.R. no comparison can be made at this level since the Task 
Force estimates are for the national level only. 

The only Task Force estimate made available in October was that for total 
wheat production.,. Therefore, the October LACIE estimates are compared to the 
September Task Force estimates. 

The LACIE winter wheat area estimate characteristically increases during the 
late season due to the confusion of hay and row crops with small grains. 

This inability to differentiate between small grains and other crops is caused 
primarily by the use of single acquisitions from specific time periods during 
the growing season. To avoid this confusion the thresholding procedure 
described in section 1.3 was utilized by CAS. 

7.1 PRODUCTION ESTIMATES 

The LACIE and Task Force estimates of production for winter, spring, and 
total wheat are shown in table 7-1. The LACIE August estimates for spring 
wheat do not contain the estimates for the regions of Tyumen and the North- 
west since no usable acreage data existed for these regions. For September 
and October there were no LACIE estimates for Tyumen and the Northwest but 
the estimates given in the CMR's (and shown in table 7-1) include an estimate 
based on historic data for these regions. The August estimates for spring 
and total wheat are significantly different from the respective Task Force 
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TABLE 7-1 PRODUCTION 



Number of 
segments 
(n/M) 

U.S.S.R. 

Task 

Force 

estimate 

(MT 10^) 

LAC IE 

RD® 

(%) 

Value 

of 

test 

statistic 

Estimate 
{MT X 10^) 

CV 

(%) 

August 5, 1977 

Winter 

658/1157 

60.0 

63.0 

4.4 

4.8 

1.09*^ 

Spring 

491/1412 

45.0 

34.6 

9.2 

-30.1 

-3.27* 

Total 

1149/2569 

100.0 

97.6 

4.3 

-7.6 

-1 .77* 

September 7, 1977 

Winter 

713/1157 

60.0 

63.9 

4.3 

6.1 

mm 

Spring 

782/1416 

40.0 

37.9 

7.2 

-5.5 

Bh 

Total 

1495/2573 

100.0 

101.8 

3.8 

1.8 

HSI 



October 5, 1977 




Winter 

553/1149 

60.0 

60.8 

4.6 

1.3 

0.3^ 

Spring 

899/1 377 

40.0 

38.3 

7.0 

-4.4 

-0.6*^ 

Total 

1452/2526 

100.0 

99.1 

3.9 

-0.9 

- 6 . 2 ^ 


®RD = Relative difference = ^ LAC IE x loo]%. 

*^The total wheat average yield is derived as the software in use does 
not produce a yield for total wheat. 

^Data are not available. 

\acIE estimate is not significantly different from U.S.S.R. Task 
Force estimate at the 10-percent level. 

*LACIE estimate is significantly different from U.S.S.R. Task Force 
estimate at the 10-percent level. 
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counterparts. However, with Inclusion of this historic estimate of production 
in these two regions, the production estimates for spring and total wheat are 
not significantly different from the corresponding Task Force estimates at the 
10-percent level. There is no significant difference between the LACIE and 
Task Force production estimates for winter, spring, or total wheat released 
in the September and October CAS reports. 

The relative difference for total wheat production estimates decreased in mag- 
nitude in each successive month, from -7.6 percent in August to -0.9 percent 
in October. 

Since the difference between the latest LACIE and Task Force total wheat pro- 
duction estimates is not statistically significant, it would be reasonable to 
assume that the bias in the production estimate is very small. With a CV 
of 3.9 percent and a negligible bias, the LACIE at-harvest production estimate 
satisfies the 90/90 criterion. 

7.2 AREA ESTIMATES 

The August, September, and October LACIE and Task Force area estimates are 
shown in table 7-2.' As in the case of production, the LACIE August estimate 
for spring wheat does not include an estimate for the Tyumen and Northwest 
regions but the September and October LACIE estimates do include an estimate 
for these regions based on historical data. The LACIE winter and total wheat 
estimates for all three months are significantly different (at the 10-percent 
level) from the corresponding Task Force estimates. The difference’ for spring 
wheat was significant in August but not in September or October. However, if 
the historical area estimates of 0.7 million hectares (1.75 million acres) 
for the Tyumen and No.-'thwest regions were added to the August spring wheat 
area estimates, the difference between the LACIE and Task Force area estimates 
for spring and total wheat would not be significant at the 10-percent level. 
The CV of the area estimate for each type of wheat in every month is small 
indicating the high degree of dependability in the area estimate. 
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TABLE 7-2 - AREA 



Number of 
segments 
(n/M) 

U.S.S.R. 

Task 

Force 

estimate 

(ha X 10®) 

LAC IE 

RD^ 

Val ue 
of 


Estimate 
(ha X 10®) 

CV 

{%) 

{%) 

test 

statistic 

August 5, 1977 

Winter 

658/1157 


24.3 

2.7 

9.5 

3.5* 

Spring 

491/1412 

mam 

38.9 

4.3 

-8.0 

-1.9* 

-0.5*^ 

Total 

1149/2569 

64.0 

63.2 

2.8 

-1.3 

September 7, 1977 

Winter 

713/1157 

20.8 

24.6 

2.7 

15.4 


Spring 

782/1416 

41.2 

41.0 

2.9 

-0.5 

Total 

1495/2573 

62.0 

65.6 

2.1 

5.5 




October 5, 1977 




Winter 

553/1149 

20.8 


3.3 

8.0 


Spring 

899/1377 

41.2 


2.6 

3.3 

Total 

1452/2526 

62.0 

65.2 

2.0 

5.1 



®RD = relative difference = LACIE^ x 100j)%. 

^The total wheat average yield is derived as the software in use does 
not produce a yield for total wheat. 

^Data are not available. 

\aCIE estimate is not significantly different from U.S.S.R. Task 
Force estimate at the 10-percent level. 

*LACIE estimate is significantly different from U.S.S.R. Task Force 
estimate at the 10-percent level. 
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Moreover, a CV of less than 3 percent for the spring wheat area estimate with 
an insignificant difference between the LACIE and Task Force estimates indi- 
cates that the LACIE spring wheat area estimate supports the 90/90 criterion. 

7.3 YIELD ESTIMATES 

The LACIE and Task Force estimates of yield for the U.S.S.,R. are shown in 
table 7-3. The LACIE winter wheat estimate was not significantly different 
(at the 10-percent level) from the corresponding Task Force estimate in the 
August CMR, but the difference was significant in the September and October 
CMR's. The LACIE yield estimate for winter wheat was low in every month. 

This was due partly to the effect of area overestimates in the low-yield 
regions, giving more weight to these regions and thus biasing the (weighted) 
average yield. 

The LACIE and Task Force spring wheat yield estimates were significantly dif- 
ferent in August but not in September or October. -The LACIE spring wheat 
yield estimate of 8.9 quintals/hectares in August is the second lowest yield 
of. the decade; however, this yield estimate is based largely on early-season- 
meteorological data obtained prior to June.- These data indicate soil moisture 
shortages in major parts of the spring wheat region which have since been . 
alleviated to some extent. 
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TABLE 7-3.- YIELD 


Wheat 

crop 

type 

U.S.S.R. 

Task 

Force 

estimate 

(ql/ha) 

LAC IE 

RD^ 

(%} 

Value 

of 

test 

statistic 

Estimate 
(ql/ha) ■ 

cv 

(%) 



August 5, 

1977 



jjfflQIlll 

27.0 

25.9 

3.4 

-4.2 

-1.2" 


11.0 

8.9 

8.7 

-23.6 

-2.7* 


16.0 

16.4*’ 

c 

-3.9 

c 

September 7, 1977 

Winter 

28.8 

26.0 

3.6 

-10.8 

-3.0* 

Spring 

9.7 

9.3 

7.1 

-4.3 

-0.6*^ 

Total 

16.1 

15.5*^ 

c 

-3.9 

c 

‘ 


October 5, 

1977 



1M 

28.8 

26.8 

3.6 

-7.5 

-2.1* 

mgm 

9.7 

9.0 

6.9 

1 

bo 

-1.1^ 

191 

16.1 

15.2*^ 

c 

-5.9 

c 


®RD = relative difference = [acie^ 

^The total wheat average yield is derived as the software in use 
does not produce a yield for total wheat. 

^Data are not available. 

\acIE estimate is not significantly different from U.S.S.R. Task 
Force estimate at the 10-percent level. 

*LACIE estimate is significantly different from U.S.S.R. Task 
Force estimate at the 10-percent level. 
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APPENDIX A 


PHASE III ACCURACY ASSESSMENT METHODOLOGY 
A.l INTRODUCTION 

This appendix contains mathematical details of the techniques use< 
in accuracy assessment. The methods used in comparing the LACIE 
estimates for acreage/ yield/ and production with the reference 
standard are presented in section A. 2. The techniques used to 
study errors in the LACIE estimates are discussed in section A. 3. 

A. 2 COMPARISON OF LACIE ESTIMATES WITH REFERENCE STANDARDS 

The reference standards to which the LACIE estimates are compared 
are the USDA/SRS estimates for. the United States and the USDA/FAS 
estimates for foreign coiintries. The statistic used for making 
these comparisons is the relative difference (RD) defined as 
follows : 

j X 100% 

where LACIE stands for the LACIE estimate of wheat production, 
area, or yield and STANDARD represents the corresponding reference 
standard estimate. This definition expresses the difference be- ' 
tween the two estimates as a percentage of the LACIE estimate. 

Significance tests of no difference are made only at the region 
or country level for the LACIE production, area, and yield esti- 
mates for spring wheat, winter wheat, and total wheat. For a sig- 
^ifiufiuce test, the XiACIE estimate (of wheat production, area, or 
yield) is assumed to be normally distributed with unknown mean u 
and variance A test of the hypothesis 

; U = STANDARD 

versus the alternative hypothesis 

1 M STANDARD 


RD 


-i 


LACIE - STANDARD 
LACIE 


A-1 



is then made using this assumption. The test statistic is given 


by 

2 - IACIE — STANDARD (A-1) 

^LACIE 


which, under the null hypothesis, is approximately normally dis- 
tributed with mean 0 and variance 1. The null hypothesis is 
j^gjected in favor of the alternative at the cx— level of significance 
if 


|Z 


> z 


a/2 


where ~ f") point of the standard noarmal 

distribution. For a = 0.10, ~ 1.645, and if jz| > 1.645, it 

is concluded that the mean of the LACIE estimator is significantly 
different from the reference standard estimate. 


A. 3 ERROR SOURCES IN LACIE 

The techniques used to study errors in the estimates of acreage, 
yield, and production are discussed- -respectively in sections A. 3.1, 
A. 3. 2, and A. 3. 3 of this appendix- 

A. 3.1 ACREAGE 

This section contains a description of the methods used to esti- 
mate the following: 

a. The errors in segment wheat proportion estimates (section 
A. 3. 1.1) 

b. Wheat acreage at the state and higher levels (section A. 3.1.2) 
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c. The variance of the wheat acreage estimates (section A. 3. 1.3) 

d. ‘ The bias in the acreage estimates for large areas having ground 

truth available for a subset of their LAC IE segments (section 
A. 3.1.4) 

e. The relative variances, of the sampling and classification 
errors in stratum wheat acreage estimates (section A. 3. 1.5) 

A. 3.1.1 Error in Proportion Estimates at the Segment Level 

This section describes the statistical calculations used to com- 
pare CAMS wheat proportion estimates for blind sites with the 
corresponding ground truth values. Let N be the number of seg- 
ments allocated to a region (state or higher level) and let n be 
the number of blind sites selected randomly from these N segments. 
For a region, let represent the CAMS estimate of the proportion 
of wheat in the xth segment and let represent the ground truth 
proportion of wheat in the ith segment, where 'i = 1, ..., N. 

Then the average error Up is given' by 


y 


D 




(A-2) 


'The estimate of is given by 


D = 




(A-3) 


where the summation is taken over the n blind sites. 
= X^ - we may estimate the variance of D by 



Letting 


(A-4) 


Lower and upper confidence limits for the population average dif- 
erence are given by 



D 


^l-a/2 


S_ 

D 



D + t 


l-a/2 


S_ 

D 


(A-5) 
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where value of the l-a/2 percentage point, from the 

Student's t distribution with (n-1) degrees of freedom, correspond- 
ing to the desired confidence level of 1-a. 


The hypothesis = 0 (i.e., no bias) is rejected at the a-level 


of significance if D/S 


D 


^l-a/2' equivalently, if the con- 


fidence interval given by equation (A-5) does not contain zero. 


A, 3.1.2 Acreage Estimation 

This section gives a brief summary of the methods used to estimate 
wheat acreage. These methods are described in detail in appen- 
dix B of the CAS Requirements Docximent.* 

A. 3. 1.2.1 Background of Sample Allocation 

The- LACIE sample allocation in the U.S. Great Plains (USGP) region 
is based upon a two-stage stratified sampling scheme in which 
counties represent the primary sampling units (substrata) and 
5- X 6-nautical-mile segments are secondary sampling units. The 
criterion for determining the total sample size was the ability 
to achieve a sampling error -of 2 percent or less for the country 
wheat acreage estimates. 

Sample segments were allocated to the counties based on relative 
weights derived from agriculture and wheat acreage reported in 
1969 agriculture census statistics. Depending upon the relative 
weights, counties were designated as Group I (at least one sample 
segment in the county) , Group ii (at most one sample segment in a 
county), or Group III (no sample segments in the county). All 
Group II counties in a CRD (stratum) were combined to determine 
the number of segments allocated to the Group II part of the CRD. 


*^Crop Assessment Subsystem (CAS) Requirements Vol IV (Rev. B) 
(Change Notice, March 8, 1977), JSC-11329, LACIE C00200. 

Cn this appendix any reference to the CAS Requirements Document 
Indicates this specific doctiment. 
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A probability proportional to size {PPS) procedure was applied to 
select the Group II counties in a CRD which were to receive these 
segments. 

Once the number of segments to be allocated to each county was 
determined, the sample segments were selected at random within the 
agricultural area of the county. For further details of the LACIE 
sampling scheme refer to the CAS Requirements Document (JSC-11329) 

A. 3. 1.2. 2 Aggregation of Acreage Estimates 

Wheat acreage estimates are made for each CRD, state, and region 
(group of states) in the USGP. However, no estimate is made for 
a state if it does not contain three or more segments satisfactor- 
ily processed by CAMS, Segment data may be lost due to the fol- 
lowing cases of nonresponse: 

a. The sample segment being obscured by cloud cover . 

b. Landsat data quality being insufficient to permit processing 

c. Landsat data acquisition failing to register with the refer- 
ence Landsat image 

d. Failure of acquisition/processing procedures to provide an 
acceptable estimate 

No replacement is allowed if a sample segment is not workable by 
CAMS. 

A CRD acreage estimate consists of three components; 

1. An acreage estimate for the Group I counties in the CRD for 
which segment data exist, (A Group I county is treated as a 
Group III county if it does not have at least one segment with 
an acceptable proportion estimate,) 

2. An acreage estimate for the entire set of Group II counties 

in the CRD if there is at least one segment with an acceptable 
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proportion estimate in this set of counties. (Otherwise, the 
Group II counties are all treated as Group III counties.) 

3. An acreage estimate for the Group III counties, including the 
Group I and Group II counties being treated as Group III 
counties. 

The wheat acreage estimates for these three components are com- 
puted using a stratified random sampling estimator for the Group I 
counties, a PPS estimator for the Group II counties, and a ratio 
estimator for the Group III counties.* 

There are three categories of Group III acreage estimates, depend- 
ing on the number of segments in a CRD for which data are available 
Categories 1, 2, and 3 correspond respectively to three or more 
segments, one or two segments, and no segments having data avail- 
able. The ratio used for the Group III estimator is the ratio of 
historical wheat acreages for Group III counties to Group I and 
Group II counties. For category 1 estimates the ratio is based 
on historical acreages in the CRD. For category 2 and category 3 
estimates the ratio is based on acreages in the state containing 
the CRD for which the estimate is being made. 

The CRD wheat acreage estimate is obtained from the sum of the 
wheat acreage estimates for Group I, II, and III counties. Next, 
aggregation of the CRD acreage estimates gives a state wheat acre- 
age estimate, and summation of the state acreage estimates gives 
the regional wheat acreage estimate. For specific aggregation 
formulas, see appendix B in the Cas Requirements Dociament. 

In a mixed wheat area, separate aggregations are performed for 
spring and winter wheat and the total wheat acreage estimate is 
obtained by summing the results. This is done at the CRD and 
higher levels. 

*For details on these standard estimation procedures, see Sampling 

T<=>nhm* mipi?? Hv W.n. Hochran , WtIrv. 





A. 3.1.3 Acreage Variance Estimation 

The acreage variance estimation for a CRD requires an estimate of 
within~county variance for each of the Group I and Group II coun-r 
ties in the CRD. Often there is only one sample segment in a 
county and hence no direct estimate of the within-county variance 
is possible. Therefore, ' an indirect method is employed. This 
method uses a regression approach and is based on the assumption 
that the historical couhty proportions are well correlated with 
the CAMS proportions. The method consists of {1) forming homo- 
geneous groups of counties in a state with respect to the within- 
county variability, (2) performing regression for the CAMS seg- 
ment wheat proportion estimate onto the county historical wheat 
proportion, and (3) taking the residual mean square error (MSB) 
for an estimate of the within-county variance for each county in 
the group . 

For estimation of a CRD acreage variance, the acreage variance 
components for Group I and Group II coxinties are estimated inde- 
pendently. For- Group I counties it is computed according to the 
variance formula for a stratified random sampling scheme. ^ The 
appropriate inputs of county sizes, number of sample segments, 
and within-county variance estimates are obtained using the above- 
mentioned procedure. Similarly, the variance formula for a PPS 
estimator^ is used to compute the Group II acreage variance 
estimate. : It requires ail of the inputs mentioned in the Group I 
case plus the probabilities of selection of Group II counties for 
sample allocation. These probabilities are those utilized in 
determining which of the Group II counties in a CRD receive sam- 
ple segments-. 

The acreage variance component for the Group III counties depends 
directly on Groups I and II variances and contributes to the CRD 

^Cf = Sampling Techniques , by W. G. Cochran, Wiley, 1963. 
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acreage variance indirectly through the ratio utilized to obtain 
the Group III acreage estimate. The foinnulas used to calculate 
the acreage variance for the Group III counties are described in 
appendix B of the CAS Requirements Doc\iment. As mentioned above ^ 
there are three categories of Group III acreage estimates and 
each category has a different formula for the variance estimate. 
For category 1 the variance estimate depends on the acreage esti* 
mates for all the Group I and Group II counties in the CRD; for 
categories 2 and 3 it depends on the acreage estimates for all oJ 
the Group I and Group II counties in the state. 

If data are available for at least three ■ segments in each CRD in 
the state, the acreage variance estimate is computed by adding 
the variance estimates for the CRD's in the state. Otherwise, 
the state variance estimate is obtained using an aggregation pro- 
cedure which accounts for the dependence between various CRD 
acreage estimates in a state. 

Since the -state, acreage estimates are obtained independently, the 
acreage variance estimates at both the regional and country leve] 
are computed by adding the state acreage variance estimates. 

In a mixed wheat area, separate aggregations are performed for 
estimating the variance of the spring and winter wheat acreage 
estimates at the CRD and higher levels. In each case the estima- 
tion procedure is the same as that described above for each aggre 
gation level. The acreage variance estimates at the CRD and 
state levels for the total wheat case are obtained from the pre- 
viously described variance formulas using total wheat acreage 
estimates for sample segments and the historical total wheat for 
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counties in the area. For higher levels the total wheat acreage 
variance, estimates are computed by taking the sum of the vari- 
ance estimates for the states involved. The CRD and state level 
variance estimates for the total wheat case are not unbiased; 
therefore, the method of determining variance of a total wheat 
acreage estimate in a mixed wheat area is considered approximate. 

A. 3.1. 4 Acreage Bias Estimation 

The, method for estimating bias described in this section is 
valid for any area having a sufficient number of blind sites to- 
represent the bias. In this report it is applied at the state 
and higher levels. 

The LACIE estimate of wheat acreage for a given area can be 
written 

n 

A = E (A-6) 

i=l ^ ^ 

where A is the estimated wheat acreage, is the wheat propor- 
tion estimate in the tth LACIE segment, n is the number of 
processed LACIE segments, and known weights based 

on historical and cartographic data.* 

/V 

Corresponding to A is the true acreage. A, which can be written 

n 

A = E W?C. (A-7) 

i=l ^ ^ 


*The precise definition of W£ depends on whether the ±th segment 
is used as part of a Group III estimate. 
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where is the true wheat acreage for the county containing 
the ith segment and»W?.is the value of the weight which would 
give perfect Group III estimates of wheat acreage for unsampled 
counties . 


We can now write 


ii = Ci + (x. - c.) + (5. - xj 


= C. + « . + e. 

where is the true wheat proportion of the ith segment, 6^ is 
the sampling error and is the classification error. Since 
sampling is unbiased,. we assume = 0? however, we do not 

assume unbiased classification. Instead, let 0 be an average 
segment bias; i.e., 

E(eJ = e 

The bias in A is defined by E (A - A) , which is thus given fay 

'' /a - ' \ 

B = E(A - A) = E 13 W.X - 13 W*C. 

\i=l ^ ^ i=l ^ V 


= (=i ^i) - jE wtc. 

- (”i - «|)Ci * e W. (A-8) 


Note that the first term of equation (A-8) represents a bias 
caused by the failure of the Group III ratios to be exact; 
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(i.e./ 7 ^ wp , whereas the, second term is the average segment 

bias multiplied by the sum of the W^. 


At present, only the second term of equation (A-8) will be 
estimated, since good county-level data are not available for 
estimating the first term. The second term is estimated by 
(1) breaking up the large area into strata (not necessarily 
connected) for which the bias is assumed to be approximately 

constant; (2) estimating 0j^ by §^. = — 2 average 

proportion error on a segment level in the ktfe stratum; and 
(3) aggregating § over- s-t-^r-ata. 

If B represents the AA estimate of bias due to classification, a 
90-percent confidence interval for 0, the real bias, can be con- 
structed by 

(B - l,64’5a, B + 1.645a) 
where is an estimate of the variance of 0 . 


If we assume Var 


(^i) ^ *^ck constant) within the “kth stratum. 


then can be estimated by 


j ^k (x. - X. - e)^ 

*2 ^ y \ 3- i L. 

4 "k - " 


n. 


and Var(-B) can be estimated by Vtr(B) = S 


where is the weight for the xth segment in the ^ith stratum. 
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A. 3.1.5 Contribution o£ Sampl ing and Classification to Acreage 
Estimation Error 

This section describes the calculation of the contribution of 
sampling and classification errors to the variance of the LACIE 
production estimate. 

A. 3 . 1 . 5 . 1 Approach 

The variance of the LACIE acreage estimate for a large area 
(e.g . , zone) can be written 

= 2 V . a? 
i ^ ^ 

where o? is the variance of the acreage estimate for the i.th 
county and is a weight which depends on the size of the 
county, the niamber of segments in the county, etc. (Refer to 
CAS Requirements Document, appendix B for details.) 

The variance represents a mean-squared deviation between the 
LACIE estimate for the county wheat proportion and the true 
county wheat proportion. This variance is caused mainly by 
two factors: sampling errors and classification errors. 

In accuracy assessment, it is desirable to quantify the contribu- 
tion of each of these error sources to the large area production 
estimate. The LACIE production estimate depends on acreage and 
yield estimation errors in a complicated way; hence, it is 
unrealistic to assume the error in the production estimate can 
be written as a sum of uncorrelated random variables representing 
acreage' and yield errors. Instead, the effect of a particular 
error source is measured by the reduction in the LACIE production 
variance which would be achieved if that source were eliminated. 
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It will be assumed (section A. 3. 1.5. 2) that the z.th county 

2 2 2 2 2 

acreage error variance can be written ^ *^ 3 ' where 

"> 2 2 . 
a is a contribution due to classification, and X a is a con- 
c» ® 

tribution due to sampling. To determine ‘the effect of no 

classification error, the variance of the IA.CIE production 

2 2 

estimate will be calculated using instead of cf^ where p is 

an estimate of the ratio — ^ — = 2~2* Similarly, the 'effect of no 

,a„ + X^a" 
c s 

2 2 

sampling error is estimated by replacing by (1 - p)cr^. This 
procedure is described in detail in section A.3.3,5 of this 
appendix. The following two sections describe the methods 
employed for estimating sampling and classification variances 
■ and the function p . 

A. 3. 1.5. 2 Acreage Regression Models 

For counties with one sample segment, the LACIE estimate of the 
i.th county' wheat proportion can be written 



= C- + e. t 5 . (A-9) 

111 

where 

= LACIE estimate of the wheat proportion in the sampled 
segment 

= true (current year) proportion of wheat in the county 
. = true proportion of wheat in the sampled segment 

= sampling error = 

6^ = classification error = 
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It will be assumed that for some reasonably large area (e.g., a 
zone) the errors and 6^ have the following properties: 

e . and 6 . are uncorrel-ated 

i 1 


E (e^) - 0 

E (6jx^) - A*X. + 0 

''N = 4 

'' Kid'll - 4 


It is also assumed that there is a linear model relating the 
current year county proportions, to the historical propor- 

tions which will be denoted by Z^; i.e., 

= a + (A-10) 

where E(C.) = 0, V(C.) = oj, Cov(C. ,e.) = Cov(C.,6.) = 0, and 

X X 11 i i X -L 

a and 3 are regression coefficients. 


From the above assumptions and definitions, three basic 
regression models are obtained; 

a. True segment proportion versus historical county propor- 
tion — from the definition of e^, 

X. = C. + e. 

X 1 1 

= a .+ ^i ^i (A-11) 


It follows that 

E(X^) = a + 3Zj_ (A-12) 

, v(Xi) = + a| (A-13) 
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b. LACIE segment proportion versus ground truth segment pro- 
portion — from the definition of 5.. 


X. = X. + 6. 

1 1 X 


It follows that 


■■e(x^IXj^) = + A*X^ + 9 


Writing X = 1 + X*, one obtains 

E(Xi[Xi) = XX^ + 0 (A-17) 

v(Xi|x.) = al (A-X8) 

LACIE segment proportion versus historical county pro- 
portion — from equations (A-12) through (A-18)^ 

^(^i) = '‘x. (®(^il='l))= ®X. (“l ®) = + 6"i) « 


(A-19) 


(^i) = '^x. ^ ''x, ('^(iil==i) = “o + 


(A-20) 


As stated previously, one would like to estimate p = 2~2 

c s 

None of the three regression models permits an estimate of 

2 - 2 2 2 

o separately from a„; i.e., one can only estimate a + u„, not 

S n S il 

2 

cjg alone. If current year county proportions were available, 
2 

Pjj could be estimated, but since this is not the case. 
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2 2/2 

cf^ + 

C I s 


will be estimated instead of p* If 


o 2 

<< a (a reasonable assumption) then p* a p, 

XI 5 


A. 3. 1.^.3 Normality Assumptions — Maximum Likelihood Estimation 
of p* 

Suppose a given zone has m blind site segments and n ordinary 
(i.e., not blind site) segments, and let the blind site segments 
be numbered 1 to m. It is assumed that ground truth wheat pro- 
portions available for the blind sites and LACIE 

estimates available for all the segments. It is 

also assumed that historical wheat proportions ^ire 

2 

available for the counties containing the segments. If << a 
so that p « p* the regression models equations {A-11 through. 
A-20) can be used to obtain 

E(Xi) = a + = Og i =!,*•», m 

E(x^lXi) = AX^ + 0 ; v(x^jx.) = i = l,-“,m 

E(Xi)= 6 + Xa + XBZ^; = X^aJ + i = m+l,m-hn 


If there is one segment per county, then the errors and 6^^ 
are independent for different values of i, and hence the likeli- 
hood function of rhe sample can ■ be written 


m 

= TT 

i=l 


(Xi,S.) 


m±n / A . 

.TT >>(xj 

i=m+l 


(A-21) 


where f(x.,X.) is the joint density of X. and X. for i = l,**»,m 

\ X X / XX 

and[h ) is the density of X^ for i = m+l,*y,m+n. 
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The function n can be written JJ f(X^,X^) 

i=l issl 


f(Xj^|Xj^) g(X^) where f(X^jX^) is the conditional density 


of given and is the density function of X^. 


If normality is assumed, TT ,r,v 0 ■» -rT 1 

li " i' i’.’ n 77 


x=l c 


exp)-- f] (X - XX - exp|- y (X - a - BZ.)^ 

. ) 2 rr^ 1 1 (cfs^ I 0^2 iO 1 1 


2 a“ 

C X=1 


2 a 2 

S X=1 


m+n . 


n ‘'‘"'i' ° ®’"T ^ E '^1 - 

i=w-i °s + °s ■" “c) i^S -1 


- 0 - XBZ^) 


Letting Q = -21ogL - log2Tr, 

D T? T 

Q = m log cTg + m log Og + n log(o^ + X^aJ} + -| + -| + ^ 

aj c~, + 

CSC s 


(A- 22 ) 


where 


“m = \ '=<i 


’’m = 2 (x^ - a - BZ.) 


Tj^ = ^ (Xj^ - la - 0 - XBz^) 

i=m+l 
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One attempts to maximize L by finding a stationary point of Q; 


2 3 a 


2 - a - BZ,) E "'^1- ® 


o‘ + \ 0 

C S 


1 3Q 
? 3 ^ 


2 Zi(x^ - a *- 3Z^) S ^^i^% - Aa - e - XBZ^) 


2 , ,2 2 
a + A a 
c s 


1 

2 30 


ni ^ 

? 'Xj. 


m+n 

{ ^ “ Xa ~ 6 

m+1 


XBZ^) 


^2 . , 2^2 

a + X ff 

c s 


1 3 Q 
T'5T 


2 X. (X . - Xx- 
1 ^ ^ ^ 


5 TO+n 

6 ) + 2 + a) (Xi - 

j i=m+l 

— 2^,22 


X a^T„ 

. s n _ n 

/„2 . >2 2\2 “ ° 

(‘'c ^ ^s) 


XV t 

s c 




m . nX 

“S’ ■*■ “T "2 2 

af X^oj + 
s sc 


/ 2 ^ .2 2 \i 

("o ^ s) 


Equations (a- 23) through (A-29) must be solved for the parameters 

2 2 -/S. -A. ^ ^2 '^2 

a, 6r 0, X, o^, and a^. If a, $, 0, X, a^z and represent the 
solution to equations (A-23) and (A-29 ) , then the invariance 
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theorem, for maximum likelihood estimation can be used to 
obtain 


P = 




%2 , 

Oc + (X) 0g 


as the maximum likelihood estimate of p. 


(A-29) 


The equations (A-23) through (A-29) are nonlinear but can be 
solved using numerical techniques: Newton’s Method was used to 

solve the equations for this report; i.e., if u^^^ is an estimate 
of the solution vector u - {S/ ^ . a^) at the ktTi step, 

then 


„(k) . 


(A-30) 


Where f(u^^^) = (f^, • • • ,fg)'^ is the vector of the left sides of 

' (kl 

equations (A-23) through (A-29) evaluated at u' and F = 


In practice, it was simpler to use the parameter transformations 

(A-31) 


r = 


~T~1 5 " 

s c 


and 


s = X^ol + 

s c 


(A-32) 


and solve for a, 3, 6, X, r, and s. Again, the invariance 
theorem can be used to give 


^ f'2'^ 

p = X r 
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A. 3. 1.5. 4 Accuracy of ^ 

Since 5 is an extremely complicated function of the data, it is 
impossible to write down the variance of p for finite sample 
sizes m and n* However, the asymptotic variance of ^ can be 
estimated using the information matrix? i.e., if 




and g(u) = g|a, 3, 0 , X , a^,Og j is a differentiable function of the 

parameter vector th$n the variance of g (u) is asymptotic to 

•• 

•[g‘ (u) v"^g' (U) 


where 


g*(u) 


\3Ui 


;; 


(A-33) 


Thus, in our case, g(u) = 


,2 2 

X- a 

s 


■>2 2 ^ 2 
"s * “o 


g’ (u) = 


0 , 0 , 0 , 2\olcl{\^al + + 0 ^) , 


-2 2 


/ 2 ^ , 2^2 \2 
^ ''s] . 


(A»34) 


To estimate V, the observations {X^}, {Y^}, and and the 

estimated parameters B,6,X,a^, and a were substituted into 

the matrix H = (h. .} = ^ equation (A-33) was used 

1 ] 0 U . 0 u . 

ta obtain an approximate variance for p. 


A. 3 . 1 . 5 . 5 Coefficients of Variation of a Large Area Estimate Due 
To Classification and Sampling Errors 

Let p be the ratio of the within-county sampling variance^ esti- 
mate to the total within-county area variance estimate as defined 


A-20 



in (A-29) . Assuming that this ratio also applies to a large area, 
the variances of the large area estimate due to classification and 
sampling are given, by 

^2 '^ 2 ' 

= (1 - p)V^ 

and 

^2 ''2 
V = pV 

^2 '^2 ''2 

where n f V , and V denote the classification variance, the 
seunpling variance, and the acreage variance for the large area 
estimate, respectively. Consequently, the estimated CV of a 

/N 

large area -estimate A due to classification is given by 

CV(A|C) = ^ 

A 

and the estimated CV of large area estimate due to sampling is 
given by 

CV(a|s) = ~ 

A 

where CV(AjC) and CV(a|s) are often casually referred to' in 
LACIE as the classification CV and sampling CV, respectively. 

A. 3.2 YIELD 

This section contains a description of the methods used to pre- 
dict yields' {section A. 3.2.1) and to estimate yield prediction 
error (section A. 3. 2.2). In Phase II no estimate of yield bias 
was made. 


A. 3.2.1 Yield Prediction 

Most of the yield predictions made in LACIE are provided by the 
Center for Climatic and Environmental Assessment (CCEA) of NOAA. . 
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They are produced from multiple linear regression yield models* 
developed on historical weather and yield data. Usually these 
models cover a state but in some cases they cover part of a state 
or part of two states and in some cases they overlap. 

In a given state there is either one yield stratum or two. In 
the first case the state yield prediction is that given by the 
CCEA model. In the second case the state yield prediction is 
given by; 

Y = P/A {A- 35) 

where P is the production estimate (section A. 3.3.1} and A is the 
acreage estimate (section A. 3. 1.2) for the state. The yield pre- 
diction at the region or country level is also obtained from 
equation (A-35) , with P and A in that case being the production 
and acreage estimates at the corresponding level. 


A. 3.2.2 Estimation of the Yield Prediction Error 


CCEA provides estimates of the mean squared yield prediction error 
at the stratum level. In the CAS Requirements Document it is 
shown that at the state, region, or country levels the estimate 
of the mean squared yield prediction error for a given area 
(state, region, or country) is 


’7 


2 




SY. V? 
1 1 

P A 


(A-36) 


where 

2 

S = estimated mean squared prediction error of the production 
estimate P for the area 

2 

V = estimated variance of the acreage estimate A for the area 

*Wheat Yield Models for the United States (LACIE 00431) , National 
Aeronautics and Space Administration, Johnson Space Center, 
Houston, Texas, June 1975. 
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= yield estimate for the 1th pseudo zone in the area 

2 ' 

= estimated variance of the acreage estimate for the 1th 
pseudo zone in the area 

In the case where there is only one yield stratum for a state, 
the yield prediction error for the state is given directly by 
the ,CCEA model. 

A. 3 . 3 PRODUCT 101 

This section contains descriptions of the methods used to do the 
:ol lowing: 

a. Estimate wheat production (section A. 3. 3.1). 

b. Estimate the variance in the wheat production estimate 
(section A. 3.3.2). 

c. Estimate the bias in the wheat production estimate 
(section A. 3, 3, 3). 

d'. Evaluate whether LACIE is satisfying the 90/90 criterion 
(section A. 3, 3. 4). 

e. Determine the effect of errors in area, yield, sampling, 
and classification on the production variance 
(section A. 3. 3. 5) . 

A. 3, 3,1 Production Estimation 

At the CRD level the production estimate is obtained by multi- 
plying the area estimate and the yield prediction for the CRD. 

I The area estimate is made for the CRD itself but the yield pre- 
'diction is made for a group of CRD's in a state (section A. 3. 2. 3 

The production estimates for the state and higher levels are 
obtained by simply adding the estimates for all the CRD's in 
the area. 
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A. 3.3.2 Production Variance Estimatiion 

Since the production estimate is the product of an acreage esti- 
mate and a yield prediction, the measure of. variability in the 
estimate should properly be called the production mean squared 
prediction error. However, in this report, for simplicity this 
quantity will be called the production variance. 

Since the yield predictions are made for a group of CRD's it is 
not possible to obtain independent production variance estimates 
at the CRD level; hence, the estimates of production variance are 
made only at the state and higher levels. The estimation proce- 
dures are described in detail in appendix B of the CAS Require- 
ments document. 


A. 3. 3. 3 Production Bias Estimation 

The production bias at the state level is given by 


Sp. - E(Pi - Pi> 




(A-37) 


= E(A.Y.) - A.Y. 


where A^, and are respectively the true values of the 

acreage, yield, and production for the Nth state in question, 

rfV -A 

and A., Y. f and P., are the corresponding estimates for these 
quantities. Assuming A^ and are independent, one obtains 


Bp = E(A^)E(Y^} - A^y^ 


(A-38) 


If one further assumes that Y. is unbiased, then E(Y. ) = Y.: / 

X XX 

(A-39) 


B_ = Y. [E(A. ) - A.] 
P^ 11 1 


= Y.B^ 
1 A . 
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where B is the acreage bias for the Lth state. The quantities 
Y. and B,- are unknown/ but an estimate B for can be 

obtained by using the estimates for Y. and described in 
sections A. 3. 2.1 and A. 3. 1.4, respectively. Thus, 


B„ == Y.B, 
P. 1 


(A-40) 


The variance of Bp is given by 


Var^Bp j = Y? Var^B^ J 
and estimated by 

j As \ . As*) As f A, \ a) a a a JA \ A A 

Var|B„ 1 = YT Var/B, I + B, Var(Y.) - VarlB. | Var(Y.) 

A 

For the nine-state level, the production bias estimate B ' is 


B. 


= Z % = E 5A, 


and the. estimate of its variance is ^ Var/B V The relative 

^ \ i/ 

bias of the production estimate R(Bp) is estimated by expressing 
the production bias as a percentage of the LACIE production esti- 
mate; that is, by 


. Z 


* . B 

R(B ) = 100% 

" ZaA 


(A-41) 


A. 3. 3. 4 Evaluating the 90/90 Criterion 

Let P be the LACIE estimate of wheat production for the region or 
cotmtry, and let P be the true wheat production of the same region 
or country. The accuracy goal of the LACIE is a 90/90 at-harvest 
criterion for wheat production, which is given by the following 
probability statement. 
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(A-42) 


Pr yp ” P{ < O.ipj >0.90 

This states that the accuracy goal is for the LACIE estimate of 
wheat production to be within 10 percent of the true .wheat pro- 
duction with a probability of at least 0.9. 


It is assumed -that the LACIE estimate, P, is normally distributed 

2 

with mean P + B and variance where B is the bias given by 

P 

B = E(P) - P 

Under this assumption, equation (A-42) may be written as 


Pr 


-0.1 - 0.9 


B 


P+B 


0.1 - 1.1 


< Z < 


B 

P+B 


CV{P) 


CV(P) 


> 0.90 


(A-43) 


P — (P+B) 

where Z = — follows the standard normal distribution, 

rs 

aP 

N{0,1), and CV(P)- is the coefficient of variation of P defined 
by 


:v(P) 


E(P) 


og 

P+B 


(A-44) 


The term _ is called the relative bias of P and is given by 

A. 

B _ E(P) - P 
P+B 

E(P) 


It follows that the accuracy goal of LACIE is ' attained if 


0.1 - 1.1 


B 


P+B 


CV(P) 


.. $ 


-0.1 - 0.9 


B 

P+B 


CV(P) 


> 0.90 


(A~45) 


where $ represents the cumulative standard normal distribution.^ 
The enclosed region of figure A-1 indicates combinations of CV(P) 
and relative bias for which equation (A-40) is satisfied. 
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COEFFICIEMT OF VARIATION, CV(P)' 



.09 .08 .07 .06 .05 .0«( .03 .02 .01 -.01 -.02 -.03 -.04 -.05 -.06 -.07 -.08 -.09 -.10 -.11 

RELATIVE BIAS 


A 

Figure A- 1 Diagram showing value of relative bias and CV(P) for which 

90/90 criterion is satisfied. 


The estimates of CV(P) are provided by CAS and the estimates of 
the relative bias are obtained using the method described in 
section A»3,3.3. 


A proper evaluation of the 90/90 criterion can only be made toward 
the end of the season since the results for spring' wheat are 
normally not available before August. In order to gauge how 
well LACIE is performing early in the season when only winter 
wheat data are available, a method of projecting the winter 
wheat results for the 5- or 7-state level to the 9-state total 
harvestable wheat level using Phase II results was developed. 

Since adequate blind site proportions are not available in the- 
early season, the relative difference between the LACIE and USDA 
estimates is taken as an estimate of the relative bias. The 
"projected" relative difference at the 9-state level is given by 
the equation 



-12.3 


^77 


RD 


76 


(A-46) 


where -12.3 is the Phase II final relative difference at the 
'9-state level, current month relative difference in 

1977 for the 5- or 7-state winter wheat production estimate, and 
RD^g is the corresponding relative difference for the same month 
in 1976. The. values for ^^^^g are given in table Arl. 


Similarly, the "projected" CV(P) at the 9-state level is given by 

. ^77 ■ 

CVg = 5 (A-47) , 

where' 5 is the Phase II final CV(P) at the 9-state level, 
is the current month CV(P> for the 5- or 7 -state winter wheat 
production estimate, and CV^g is the corresponding CV. for the same 
month in 1976. The values for CV^g are given in table A-1. The 
7-state results are used if they are available for both years. 
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TABLE A-1.- PHASE II CV'S AND RELATIVE DIFFERENCES 


Date 

Area, 

state 

■ 

' Phase'll 
CV for production 

Phase II 

relative difference 

Feb* 

5 

11 

-4.9 

Mar. 25 

' 5 ■ - 

10 

-9.9 

Apr. 8 

5 

8 

-8.5 

May 7 

5 

8 

-1.6 

June 8 

5 

7 

fll.4 


7 

8 

+1.7 

June 29 

5 

7 

fl2.7 


7 

7 

+4.7 

July 9 

5 

7 

-3,7 


7 

7 

-7.9 

Aug . 11 

5 

7 

-4.2 


7 

7 

-5.6 


9 

6 

• 

H 

1 

Sept. 9 

5 

7 

—6 • 6 


7 

7 

-6.6 


9 

5 

-13.6 

Oct. 8 

5 

7 

-6.6 


7 

7 

-6.5 


9 

5 

-13.8 

Dec . 17 

5 

7 

. -7.2 

(final) 

7 

7 

-7.2 


■ 9 

5 

-12.3 


ORIGINAL PAGE IS 
OP POOR QUALITY 
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After RD* and CVg have been calculated, inference as to whether 
■the 90/90 criterion has been satisfied can be made by determining 
whether these values fall within the enclosed area in figure A-1. 
If they do, it is said that the current LACIE estimates support 
(rather than satirsfy) the 90/90 criterion since the determination 
was based on projections which may or may not be accurate. 


A. 3. 3. 5 Effect of Errors in Acreage, Yield, Sampling, aim 
Classification on th^ Production Varianc e 

The production variance consists of two major error components; 
acreage and yield. The acreage error may be further subdivided 
into sampling and classification errors. The effect of a par- 
ticular error is determined by the reduction in the production 
variance estimate' when the error is omitted from the calculation 
of that estimate. If there is only one yield stratum -in a zone 
(state), the production variance is calculated- at the zone level 
and aggregated to higher levels. If a zone contains more than 
one yield stratum, it is subdivided into pseudozones, which are 
the intersections of the zone with the various .y.ield strata. 

The production variance estimate is then calculated at the 
pseudozone level and aggregated to the zone and higher levels. 


Suppose the zone consists of H pseudozones ^ 


G^,G2,/ 


,Gjj, with 


acreage estimates A21'‘^Z2 ^ yield predictions 






ZH' 


respectively. Then the estimate of the produc- 


tion variance at the zone level is given by the following equa- 
tion, which also appears in appendix B of the CAS Requirement 
Document. 


H 

=i:( 


2 2 2 2 


i"L). 


i=l 


H i-l 
i=2 £=-1 


Y Y 
Zi ZO 



(A-48) 
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where 


2 

= the estimate of tlio yield variance for the Lth pseudoijor-e 

2 

^Zi ” area variance estimate for the ith pseudozone 

T 

- the estimated covariance between in and A^^ in 

In order to determine the production variance without a given 
error termj, equation (A-4 8 ) must be rederived with that term 
omitted. Let and be the state production 

variances without acreage, .yield, sampling, and classification 
errors, respectively. One obtains the following expressions f 
these quantities. 

= X) <*-«> 

i=l 


H 


®ZY ' X - ''lA) 


i=l 

-eSwe £ V 

- i=2 v^^i , 


zs 


H 

X[ 


''•2 2 2 2 « 4 2 2 

(1 - P)V:,y1. + U$,a1. - (1 - 


Zi Zi • Zi“Zi 


H i“l 


' X X ^zi’^zz ( .£ .£ ’’jk') 
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(A-51) 
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H 


ZC 


= E( 


2 2 2 2 ■'2 2 
■PV^ Y* + rj* - PV 

^^Zi^Zi Zi Zi Zi 


•tJ^) 

L ZX/ 


i«l 


H i-1 


” E 2 >.A. is S 

i=2 £=i 
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Where p is defined in (A-29) . 


9 2 2 2 

Let r)£’ ^rS* ^rC regional-level production 

variance estimates without acreage/ yield/ sampling, and classi- 
fication errors, respectively. These estimates can be obtained 
from the following expressions. 




R 


rA 


= E4^i: E 


rzz* 


z=i 


z=i z*=i 
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= t 


ry Z-/ "ZY 

Z=1 


(A- 54) 


R 


R R 


rS 


= E4^EE 


rZZ 


I. 


Z=1 

R 


2=1 z'=l 

R R 


ZC 


' E E E 


rZZ' 


Z=1 


Z=1 Z'=l 
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(A~56) 


Here R is the total number of zones in the region and 3^.22* " ® 
if Zth and Z'th zones have no yield strata in common. Otherv;ise, 


^rZZ' S \zK*“rZ'K^ 


2 

rK 


(A-57) 


K=1 
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where 


^rZK ~ area estimate for the pseudozone corresponding to 
yield stratum K in zone Z of region r 

2 

= the squared prediction error for the Kth yield stratum 
conunon to zones Z and Z ‘ 

C “ the number of yield strata corrar.on to the Zth and Z'th 
zones 

The estimates of the' corresponding variances for a country are 
obtained by adding the corresponding estimates for all the 
regions in the country. These' computations assume that the 
regional production estimates are uncorrelated. 
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APPENDIX B 


PHASE III INTENSIVE TEST SITES 


To accomplish the objectives of accuracy assessment, data includ- 
ing ground truth ,, aircraft photographs-, and Landsat multispectral 
scanner imagery were gathered from 23 -intensive test sites. 
Because of factors such as a-tmospheric effects and data dropout, 
acceptable imagery was received and processed for only 17 inten- 
sive test sites and only six of these sites were processed during 
more than one biowindow. These 23 sites were located in the 
states of Idaho, Indiana, Kansas, Montana, North and South Dakota, 
Texas, and Washington (table B-1) . These states are from four 
regions; -the Nor -thwe stern United States, the Great" Lakes, and 
the Southern and Northern Great Plains. 
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TABLE B-1.- LACIE PHASE III INTENSIVE TEST SITES 


Segment 

State ^ 


Center coordinates 

Site 

size 

Wheat 

type 

(a) 

^ 0 Uil L 

N . Lat . 

W. Long* 

Statute 

mile 

Km 

196S 

N- Dakota 

Burke 

48“53.2' 

102*10.0' 

5x6 

8x9.7 

S 

1966 

N, Dakota 

Williams 

48'’19.2’ 

103*24.7' 

5x6 

8x9.7 

S 

1967 

N. Dakota 

Divide 

48°53.6' 

103*10.9' 

2x10 

3x16 

s 

1687 

S- Dakota 

Hand 1 

44°35.0' 

98*58.0' 

5x6 

8 x 9.7 

ssw 

1986 

S. Dakota 

Hand 2 

44'’21,0' 

98*45.1' 

5x6 

8x9.7 

S5W 

1969 

Montana 

Toole 

48°53.0' 

111*46.5' 

2x10 

3x16 

saw 

1970 

Montana 

. Liberty 

48*44.0' 

110*51.0' 

2x10 

3x16 

s§w 

1971 

' Montana 

Hill 

48*42.0' 

109*55.0' 

2x6 

3x9.7 

S5W 

1973 

Washington 

Whitman 

46*50.4' 

117*48.3' 

3x3 

4. 8x4. 8 

s§w 

1975 

Idaho 

Oneida 

42*04.5' 

112*29.5' 

3x3 

4. 8x4. 8 

s^w 

1976 

Idaho 

Franklin 

42*08.0’ 

111"58:0’ 

3x3 

4. 8x4. 8 

saw 

1977 

Idaho 

Bannock 

42*56.5’ 

112*25.5' 

3x3 

4.8x4. 8 

saw 

1978 

Texas 

Randall 

55*09.5’ 

102*04.4' 

3x3 

4.8x4. 8 

w 

1979 

Texas 

Deaf Smith 

34*52.2' 

102*22.3' 

3x3 

418x4.8 

w 

1980 

Texas 

Oldham 

35*15.0' 

102*32.0' 

3x3 

4.8x4, 8 

w 

1981 

Indiana 

Shelby 

39*27.6' 

85*47.2' 

3x3 

4. 8x4. 8 

w 

1982 

Indiana 

Madison 

40*13,5’ 1 

85*37.5' 

3 x 3 

4. 8 x 4 . 8 

w 

1983 

Indiana 

Boone 

40*05.7’ 

86*33.5' 

3x3 

4. 8x4. 8 

w 

1960 

Kansas 

Morton 

37*16.0' 

101*54.0' 

5 x 6 

8 x 9.7 

w 

1962 

Kansas 

Sal ine 

38*41.8' 

97*28.4' 

3x3 

4 . 8 x 4 . 8 

•w 

1963 

Kansas 

Rice 

38*17.0' 

98*12.7’ 

3 x 3 

' 4.8x4. 8 

w 

1964 

Kansas 

Ellis 

38*50.1’ 

, 99*15.0’ 

3 x 3 

4 . 8 x 4 . 8 

w 

1988 

Kansas 

Finney 

38*10.2’ 

100*43.2' 

5 x 6 

8x9.7 

w 

1987 

Minnesota 

Polk 

47*49.0' 

96*41.0' 

5x6 

8x9.7 

s 


As indicated by ground truth:’ S = spring wheat; W = winter wheat; SW = spring and winter wheat- 
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METHOD OF DESIGNATING SEGMENTS AS SPRING, WINTER, OR MIXED 


The USDA/SRS winter wheat and spring wheat production , estimates 
for each county in South Dakota and Montana for the years 1965 
to 1976 were taken ’into consideration to determine the county 
contribution to the state total production for each crop type- 
A county- to- state contribution threshold of 1 percent was taken 
for each crop type. If a county, containing allocated segments, 

contributed 1 percent or more to the state winter wheat produc- 

/ 

tion, its segments were designated as winter — similarly for 
spring wheat. This divided the counties into three groups: pure 

spring, pure winter, and mixed. Further, those counties in the 
pure spring and pure winter groups were then designated mixed if 
the within county contribution for either crop type to the total 
wheat for the county was between 25 and 75 percent. For example, 
a county may have contributed more than 1 percent to state winter 
wheat production but less than 1 percent to state spring wheat 
production. However, spring wheat could make up 50 percent of 
the county's total wheat production. In this case, the county 
is designated as mixed. The resulting segment designations are 
in the following tables. In the group of segments not to be used, 
those that are asterisked are to be processed by CAMS as mixed 
segments for evaluation purposes but are not to be used in the 
aggregations . 


SOUTH DAKOTA 



Mixed 

Segments 


Spring 

Wheat 

Segments 

1666 

1670 

1698 

1805 

1665 

1673 

1498 

1548 

1690 

1485 

1676 

1687 

1808 

1484 

1674 

1679 

1681 

1784 

1486 

1677 

1688 

1697 

1667 

1675 

1499 

1599 


1668 

148,8 

1699 


1487 

1489 

1525 

1755 


1669 

1686 

1689 


1671 

1678 

1680 

1756 
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winter Wheat Segments 


Segments Not To Be Used 


1597 

1804 

1800* 

1809 

1683 

1694 

1801 

1811* 

1598 

1806 

1802* 

1812 

1803 

1696 

1807* 

1813 






MONTANA 




Mixed 

Segments 



Spring 

Wheat Segments 

1528 

1735 

1936 

1741 

1941 ’ 

1532 

1943 1546 

1529 

1933 

1737 

1939 

1539 

1533 

1543 1547 

1929 

1934 

1937 

1534 

1540 

1940 

1544 1945 

1732 

1736 

1738 

1535 

1942 

1541 

1944 1946 

1932 

1935 

1739 

1536 

1555 

1542 

• 1545 1559 

1733 

1530 

1938 

1537 




1734 

1531 

1740 

1538 






Winter 

Wheat 

Segments 


Segments Not 

To Be 

Used 

1725 

1930 

1745 

1549 

1552 

1928 

1553 


1728 

1931 

1948 

1550 

1556 

1947 

1103 


1729 

1742 

1747 

1753 

1557 

1551 

1554 


1730 

1743 

1750 

1101 

1104 

1752* 



1731 

1744 

1949 

1102 

1558 




*To : 

be proces 

sed by 

CAMS as 

mixed 

for evaluation but not 

for 


aggregation. 
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